Broadcasting signal transmission device, broadcasting signal reception device, broadcasting signal transmission method, and broadcasting signal reception method

ABSTRACT

A method and device for processing a broadcast content are discussed. The method includes receiving a broadcast content of a broadcast service; displaying the broadcast content on a display screen; and detecting a first user input, in response to the first user input that is to display an ESG: displaying the ESG overlaid on the broadcast content being displayed on the display screen; determining whether information for controlling execution of an application, that is embedded in an audio component of the broadcast content, is received; and controlling the execution of the application based on the determination; or in response to the first user input that indicates a mute mode to mute sound of the broadcast content: determining whether information for controlling execution of the application, that is embedded in a video component of the broadcast content, is received; and controlling the execution of the application based on the determination.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Divisional Application of U.S. patent applicationSer. No. 15/579,482, filed on Dec. 4, 2017, which is the National Phaseof PCT International Application No. PCT/KR2016/007278, filed on Jul. 6,2016, which claims priority under 35 U.S.C. 119(e) to U.S. ProvisionalApplication Nos. 62/189,187, filed on Jul. 6, 2015, 62/202,180, filed onAug. 7, 2015, 62/207,356, filed on Aug. 19, 2015, 62/213,626 filed onSep. 3, 2015 and 62/214,181 filed on Sep. 3, 2015, all of which arehereby expressly incorporated by reference into the present application.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an apparatus for transmitting abroadcast signal, an apparatus for receiving a broadcast signal andmethods for transmitting and receiving a broadcast signal.

Discussion of the Related Art

As analog broadcast signal transmission comes to an end, varioustechnologies for transmitting/receiving digital broadcast signals arebeing developed. A digital broadcast signal may include a larger amountof video/audio data than an analog broadcast signal and further includevarious types of additional data in addition to the video/audio data.

That is, a digital broadcast system can provide HD (high definition)images, multichannel audio and various additional services. However,data transmission efficiency for transmission of large amounts of data,robustness of transmission/reception networks and network flexibility inconsideration of mobile reception equipment need to be improved fordigital broadcast.

SUMMARY OF THE INVENTION

The present invention provides a system capable of effectivelysupporting future broadcast services in an environment supporting futurehybrid broadcasting using terrestrial broadcast networks and theInternet and related signaling methods.

The present invention can effectively support future broadcast servicesin an environment supporting future hybrid broadcast using terrestrialbroadcast networks and the Internet. In addition, the present inventionproposes architectures and schemes for efficiently using video WM andaudio WM.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIG. 1 is a diagram showing a protocol stack according to an embodimentof the present invention;

FIG. 2 is a diagram showing a service discovery procedure according toone embodiment of the present invention;

FIG. 3 is a diagram showing a low level signaling (LLS) table and aservice list table (SLT) according to one embodiment of the presentinvention;

FIG. 4 is a diagram showing a USBD and an S-TSID delivered through ROUTEaccording to one embodiment of the present invention;

FIG. 5 is a diagram showing a USBD delivered through an MMT according toone embodiment of the present invention;

FIG. 6 is a diagram showing link layer operation according to oneembodiment of the present invention;

FIG. 7 is a diagram showing a link mapping table (LMT) according to oneembodiment of the present invention;

FIG. 8 is a diagram showing a structure of a broadcast signaltransmission device of a next-generation broadcast service according toan embodiment of the present invention;

FIG. 9 is a writing operation of a time interleaver according to anembodiment of the present invention;

FIG. 10 is a block diagram of an interleaving address generatorincluding a main-PRBS generator and a sub-PRBS generator according toeach FFT mode, included in the frequency interleaver, according to anembodiment of the present invention;

FIG. 11 is a block diagram showing a network topology according to anembodiment of the present invention;

FIG. 12 is a block diagram showing a watermark-based network topologyaccording to an embodiment of the present invention;

FIG. 13 is a ladder diagram showing a data flow in the watermark-basednetwork topology according to an embodiment of the present invention;

FIG. 14 is a block diagram showing a structure of a fingerprint basedimage display device according to another embodiment of the presentinvention;

FIG. 15 is a block diagram showing a structure of a watermark-basedimage display device according to another embodiment of the presentinvention;

FIG. 16 is a diagram illustrating data which can be delivered throughwatermarking according to an embodiment of the present invention;

FIG. 17 is a diagram illustrating the meaning of each value of atimestamp type field according to an embodiment of the presentinvention;

FIG. 18 is a diagram illustrating the meaning of each value of a URLprotocol type field according to an embodiment of the present invention;

FIG. 19 is a sequence diagram illustrating a procedure of processing theURL protocol type field according to an embodiment of the presentinvention;

FIG. 20 is a diagram illustrating the meaning of each value of an eventfield according to an embodiment of the present invention;

FIG. 21 is a diagram illustrating the meaning of each value of adestination type field according to an embodiment of the presentinventions;

FIG. 22 is a diagram illustrating a data structure to be inserted into aWM according to embodiment #1 of the present invention;

FIG. 23 is a sequence diagram illustrating a procedure of processing thedata structure to be inserted into a WM according to embodiment #1 ofthe present invention;

FIG. 24 is a diagram illustrating a data structure to be inserted into aWM according to embodiment #2 of the present invention;

FIG. 25 is a sequence diagram illustrating a procedure of processing thedata structure to be inserted into a WM according to embodiment #2 ofthe present invention;

FIG. 26 is a diagram illustrating a data structure to be inserted into aWM according to embodiment #3 of the present invention;

FIG. 27 is a diagram illustrating a data structure to be inserted into aWM according to embodiment #4 of the present invention;

FIG. 28 is a diagram illustrating a data structure to be inserted intothe first WM in embodiment #4 of the present invention;

FIG. 29 is a diagram illustrating a data structure to be inserted intothe second WM in embodiment #4 of the present invention;

FIG. 30 is a sequence diagram illustrating a procedure of processing thedata structure to be inserted into a WM according to embodiment #4 ofthe present invention;

FIG. 31 is a diagram illustrating a structure of a watermark-based imagedisplay device according to another embodiment of the present invention;

FIG. 32 is a diagram illustrating a structure of a watermark payloadaccording to another embodiment of the present invention;

FIG. 33 is a diagram illustrating a modification of a watermark payloadstructure using service/content information according to an embodimentof the present invention;

FIG. 34 is a diagram illustrating a modification of a watermark payloadstructure using an NSC field according to an embodiment of the presentinvention;

FIG. 35 is a diagram illustrating a watermark payload structure forlinking between video-audio watermarks according to an embodiment of thepresent invention;

FIG. 36 is a diagram illustrating an operation using linked video-audiowatermarks according to an embodiment of the present invention;

FIG. 37 is a diagram illustrating a video WM payload format according toan embodiment of the present invention;

FIG. 38 is a diagram illustrating a receiver operation when a userdisplays an ESG on a screen while a visible application is executedaccording to an embodiment of the present invention;

FIG. 39 is a diagram illustrating a receiver operation when a visibleapplication is executed after an ESG is displayed on a screen accordingto an embodiment of the present invention;

FIG. 40 is a diagram illustrating a receiver operation when a receiveris muted by a user while an audible application is executed according toan embodiment of the present invention;

FIG. 41 is a diagram illustrating a receiver operation when an audibleapplication executed after a receiver is muted according to anembodiment of the present invention;

FIG. 42 is a diagram describing advantages of video WM including audioWM according to an embodiment of the present invention;

FIG. 43 is a diagram describing advantages of video WM including audioWM according to another embodiment of the present invention;

FIG. 44 is a diagram illustrating a wm_message( ) format according toanother embodiment of the present invention;

FIG. 45 is a diagram illustrating an audio WM payload structureaccording to another embodiment of the present invention;

FIG. 46 is a diagram illustrating a structure of an EA_message( )according to an embodiment of the present invention;

FIG. 47 is a diagram illustrating an audio WM payload structureaccording to another embodiment of the present invention;

FIG. 48 is a diagram illustrating a structure of adynamic_event_message( ) according to an embodiment of the presentinvention;

FIG. 49 is a diagram illustrating a recovery file format according to anembodiment of the present invention;

FIG. 50 is a diagram illustrating a recovery file format according toanother embodiment of the present invention;

FIG. 51 is a diagram illustrating a broadcast content processing methodaccording to an embodiment of the present invention; and

FIG. 52 is a diagram illustrating a broadcast content processing deviceaccording to an embodiment of the present invention

DETAILED DESCRIPTION OF THE EMBODIMENTS

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings. The detailed description, which will be given below withreference to the accompanying drawings, is intended to explain exemplaryembodiments of the present invention, rather than to show the onlyembodiments that can be implemented according to the present invention.The following detailed description includes specific details in order toprovide a thorough understanding of the present invention. However, itwill be apparent to those skilled in the art that the present inventionmay be practiced without such specific details.

Although the terms used in the present invention are selected fromgenerally known and used terms, some of the terms mentioned in thedescription of the present invention have been selected by the applicantat his or her discretion, the detailed meanings of which are describedin relevant parts of the description herein. Furthermore, it is requiredthat the present invention is understood, not simply by the actual termsused but by the meanings of each term lying within.

The present invention provides apparatuses and methods for transmittingand receiving broadcast signals for future broadcast services. Futurebroadcast services according to an embodiment of the present inventioninclude a terrestrial broadcast service, a mobile broadcast service, anultra high definition television (UHDTV) service, etc. The presentinvention may process broadcast signals for the future broadcastservices through non-MIMO (Multiple Input Multiple Output) or MIMOaccording to one embodiment. A non-MIMO scheme according to anembodiment of the present invention may include a MISO (Multiple InputSingle Output) scheme, a SISO (Single Input Single Output) scheme, etc.

FIG. 1 is a diagram showing a protocol stack according to an embodimentof the present invention.

A service may be delivered to a receiver through a plurality of layers.First, a transmission side may generate service data. The service datamay be processed for transmission at a delivery layer of thetransmission side and the service data may be encoded into a broadcastsignal and transmitted over a broadcast or broadband network at aphysical layer.

Here, the service data may be generated in an ISO base media file format(BMFF). ISO BMFF media files may be used for broadcast/broadband networkdelivery, media encapsulation and/or synchronization format. Here, theservice data is all data related to the service and may include servicecomponents configuring a linear service, signaling information thereof,non real time (NRT) data and other files.

The delivery layer will be described. The delivery layer may provide afunction for transmitting service data. The service data may bedelivered over a broadcast and/or broadband network.

Broadcast service delivery may include two methods.

As a first method, service data may be processed in media processingunits (MPUs) based on MPEG media transport (MMT) and transmitted usingan MMT protocol (MMTP). In this case, the service data delivered usingthe MMTP may include service components for a linear service and/orservice signaling information thereof.

As a second method, service data may be processed into DASH segments andtransmitted using real time object delivery over unidirectionaltransport (ROUTE), based on MPEG DASH. In this case, the service datadelivered through the ROUTE protocol may include service components fora linear service, service signaling information thereof and/or NRT data.That is, the NRT data and non-timed data such as files may be deliveredthrough ROUTE.

Data processed according to MMTP or ROUTE protocol may be processed intoIP packets through a UDP/IP layer. In service data delivery over thebroadcast network, a service list table (SLT) may also be delivered overthe broadcast network through a UDP/IP layer. The SLT may be deliveredin a low level signaling (LLS) table. The SLT and LLS table will bedescribed later.

IP packets may be processed into link layer packets in a link layer. Thelink layer may encapsulate various formats of data delivered from ahigher layer into link layer packets and then deliver the packets to aphysical layer. The link layer will be described later.

In hybrid service delivery, at least one service element may bedelivered through a broadband path. In hybrid service delivery, datadelivered over broadband may include service components of a DASHformat, service signaling information thereof and/or NRT data. This datamay be processed through HTTP/TCP/IP and delivered to a physical layerfor broadband transmission through a link layer for broadbandtransmission.

The physical layer may process the data received from the delivery layer(higher layer and/or link layer) and transmit the data over thebroadcast or broadband network. A detailed description of the physicallayer will be given later.

The service will be described. The service may be a collection ofservice components displayed to a user, the components may be of variousmedia types, the service may be continuous or intermittent, the servicemay be real time or non real time, and a real-time service may include asequence of TV programs.

The service may have various types. First, the service may be a linearaudio/video or audio service having app based enhancement. Second, theservice may be an app based service, reproduction/configuration of whichis controlled by a downloaded application. Third, the service may be anESG service for providing an electronic service guide (ESG). Fourth, theservice may be an emergency alert (EA) service for providing emergencyalert information.

When a linear service without app based enhancement is delivered overthe broadcast network, the service component may be delivered by (1) oneor more ROUTE sessions or (2) one or more MMTP sessions.

When a linear service having app based enhancement is delivered over thebroadcast network, the service component may be delivered by (1) one ormore ROUTE sessions or (2) zero or more MMTP sessions. In this case,data used for app based enhancement may be delivered through a ROUTEsession in the form of NRT data or other files. In one embodiment of thepresent invention, simultaneous delivery of linear service components(streaming media components) of one service using two protocols may notbe allowed.

When an app based service is delivered over the broadcast network, theservice component may be delivered by one or more ROUTE sessions. Inthis case, the service data used for the app based service may bedelivered through the ROUTE session in the form of NRT data or otherfiles.

Some service components of such a service, some NRT data, files, etc.may be delivered through broadband (hybrid service delivery).

That is, in one embodiment of the present invention, linear servicecomponents of one service may be delivered through the MMT protocol. Inanother embodiment of the present invention, the linear servicecomponents of one service may be delivered through the ROUTE protocol.In another embodiment of the present invention, the linear servicecomponents of one service and NRT data (NRT service components) may bedelivered through the ROUTE protocol. In another embodiment of thepresent invention, the linear service components of one service may bedelivered through the MMT protocol and the NRT data (NRT servicecomponents) may be delivered through the ROUTE protocol. In theabove-described embodiments, some service components of the service orsome NRT data may be delivered through broadband. Here, the app basedservice and data regarding app based enhancement may be delivered overthe broadcast network according to ROUTE or through broadband in theform of NRT data. NRT data may be referred to as locally cached data.

Each ROUTE session includes one or more LCT sessions for wholly orpartially delivering content components configuring the service. Instreaming service delivery, the LCT session may deliver individualcomponents of a user service, such as audio, video or closed captionstream. The streaming media is formatted into a DASH segment.

Each MMTP session includes one or more MMTP packet flows for deliveringall or some of content components or an MMT signaling message. The MMTPpacket flow may deliver a component formatted into MPU or an MMTsignaling message.

For delivery of an NRT user service or system metadata, the LCT sessiondelivers a file based content item. Such content files may includeconsecutive (timed) or discrete (non-timed) media components of the NRTservice or metadata such as service signaling or ESG fragments. Systemmetadata such as service signaling or ESG fragments may be deliveredthrough the signaling message mode of the MMTP.

A receiver may detect a broadcast signal while a tuner tunes tofrequencies. The receiver may extract and send an SLT to a processingmodule. The SLT parser may parse the SLT and acquire and store data in achannel map. The receiver may acquire and deliver bootstrap informationof the SLT to a ROUTE or MMT client. The receiver may acquire and storean SLS. USBD may be acquired and parsed by a signaling parser.

FIG. 2 is a diagram showing a service discovery procedure according toone embodiment of the present invention.

A broadcast stream delivered by a broadcast signal frame of a physicallayer may carry low level signaling (LLS). LLS data may be carriedthrough payload of IP packets delivered to a well-known IP address/port.This LLS may include an SLT according to type thereof. The LLS data maybe formatted in the form of an LLS table. A first byte of every UDP/IPpacket carrying the LLS data may be the start of the LLS table. Unlikethe shown embodiment, an IP stream for delivering the LLS data may bedelivered to a PLP along with other service data.

The SLT may enable the receiver to generate a service list through fastchannel scan and provides access information for locating the SLS. TheSLT includes bootstrap information. This bootstrap information mayenable the receiver to acquire service layer signaling (SLS) of eachservice. When the SLS, that is, service signaling information, isdelivered through ROUTE, the bootstrap information may include an LCTchannel carrying the SLS, a destination IP address of a ROUTE sessionincluding the LCT channel and destination port information. When the SLSis delivered through the MMT, the bootstrap information may include adestination IP address of an MMTP session carrying the SLS anddestination port information.

In the shown embodiment, the SLS of service #1 described in the SLT isdelivered through ROUTE and the SLT may include bootstrap informationsIP1, dIP1 and dPort1 of the ROUTE session including the LCT channeldelivered by the SLS. The SLS of service #2 described in the SLT isdelivered through MMT and the SLT may include bootstrap informationsIP2, dIP2 and dPort2 of the MMTP session including the MMTP packet flowdelivered by the SLS.

The SLS is signaling information describing the properties of theservice and may include receiver capability information forsignificantly reproducing the service or providing information foracquiring the service and the service component of the service. Wheneach service has separate service signaling, the receiver acquiresappropriate SLS for a desired service without parsing all SLSs deliveredwithin a broadcast stream.

When the SLS is delivered through the ROUTE protocol, the SLS may bedelivered through a dedicated LCT channel of a ROUTE session indicatedby the SLT. In some embodiments, this LCT channel may be an LCT channelidentified by tsi=0. In this case, the SLS may include a user servicebundle description (USBD)/user service description (USD), service-basedtransport session instance description (S-TSID) and/or mediapresentation description (MPD).

Here, USBD/USD is one of SLS fragments and may serve as a signaling hubdescribing detailed description information of a service. The USBD mayinclude service identification information, device capabilityinformation, etc. The USBD may include reference information (URIreference) of other SLS fragments (S-TSID, MPD, etc.). That is, theUSBD/USD may reference the S-TSID and the MPD. In addition, the USBD mayfurther include metadata information for enabling the receiver to decidea transmission mode (broadcast/broadband network). A detaileddescription of the USBD/USD will be given below.

The S-TSID is one of SLS fragments and may provide overall sessiondescription information of a transport session carrying the servicecomponent of the service. The S-TSID may provide the ROUTE sessionthrough which the service component of the service is delivered and/ortransport session description information for the LCT channel of theROUTE session. The S-TSID may provide component acquisition informationof service components associated with one service. The S-TSID mayprovide mapping between DASH representation of the MPD and the tsi ofthe service component. The component acquisition information of theS-TSID may be provided in the form of the identifier of the associatedDASH representation and tsi and may or may not include a PLP ID in someembodiments. Through the component acquisition information, the receivermay collect audio/video components of one service and perform bufferingand decoding of DASH media segments. The S-TSID may be referenced by theUSBD as described above. A detailed description of the S-TSID will begiven below.

The MPD is one of SLS fragments and may provide a description of DASHmedia presentation of the service. The MPD may provide a resourceidentifier of media segments and provide context information within themedia presentation of the identified resources. The MPD may describeDASH representation (service component) delivered over the broadcastnetwork and describe additional DASH presentation delivered overbroadband (hybrid delivery). The MPD may be referenced by the USBD asdescribed above.

When the SLS is delivered through the MMT protocol, the SLS may bedelivered through a dedicated MMTP packet flow of the MMTP sessionindicated by the SLT. In some embodiments, the packet_id of the MMTPpackets delivering the SLS may have a value of 00. In this case, the SLSmay include a USBD/USD and/or MMT packet (MP) table.

Here, the USBD is one of SLS fragments and may describe detaileddescription information of a service as in ROUTE. This USBD may includereference information (URI information) of other SLS fragments. The USBDof the MMT may reference an MP table of MMT signaling. In someembodiments, the USBD of the MMT may include reference information ofthe S-TSID and/or the MPD. Here, the S-TSID is for NRT data deliveredthrough the ROUTE protocol. Even when a linear service component isdelivered through the MMT protocol, NRT data may be delivered via theROUTE protocol. The MPD is for a service component delivered overbroadband in hybrid service delivery. The detailed description of theUSBD of the MMT will be given below.

The MP table is a signaling message of the MMT for MPU components andmay provide overall session description information of an MMTP sessioncarrying the service component of the service. In addition, the MP tablemay include a description of an asset delivered through the MMTPsession. The MP table is streaming signaling information for MPUcomponents and may provide a list of assets corresponding to one serviceand location information (component acquisition information) of thesecomponents. The detailed description of the MP table may be defined inthe MMT or modified. Here, the asset is a multimedia data entity, iscombined by one unique ID, and may mean a data entity used to onemultimedia presentation. The asset may correspond to service componentsconfiguring one service. A streaming service component (MPU)corresponding to a desired service may be accessed using the MP table.The MP table may be referenced by the USBD as described above.

The other MMT signaling messages may be defined. Additional informationassociated with the service and the MMTP session may be described bysuch MMT signaling messages. [98] The ROUTE session is identified by asource IP address, a destination IP address and a destination portnumber. The LCT session is identified by a unique transport sessionidentifier (TSI) within the range of a parent ROUTE session. The MMTPsession is identified by a destination IP address and a destination portnumber. The MMTP packet flow is identified by a unique packet_id withinthe range of a parent MMTP session.

In case of ROUTE, the S-TSID, the USBD/USD, the MPD or the LCT sessiondelivering the same may be referred to as a service signaling channel.In case of MMTP, the USBD/UD, the MMT signaling message or the packetflow delivering the same may be referred to as a service signalingchannel.

Unlike the shown embodiment, one ROUTE or MMTP session may be deliveredover a plurality of PLPs. That is, one service may be delivered throughone or more PLPs. Unlike the shown embodiment, in some embodiments,components configuring one service may be delivered through differentROUTE sessions. In addition, in some embodiments, components configuringone service may be delivered through different MMTP sessions. In someembodiments, components configuring one service may be divided anddelivered in a ROUTE session and an MMTP session. Although not shown,components configuring one service may be delivered through broadband(hybrid delivery).

FIG. 3 is a diagram showing a low level signaling (LLS) table and aservice list table (SLT) according to one embodiment of the presentinvention.

One embodiment t3010 of the LLS table may include information accordingto an LLS_table_id field, a provider_id field, an LLS_table_versionfield and/or an LLS_table_id field.

The LLS_table_id field may identify the type of the LLS table, and theprovider_id field may identify a service provider associated withservices signaled by the LLS table. Here, the service provider is abroadcaster using all or some of the broadcast streams and theprovider_id field may identify one of a plurality of broadcasters whichis using the broadcast streams. The LLS_table_version field may providethe version information of the LLS table.

According to the value of the LLS_table_id field, the LLS table mayinclude one of the above-described SLT, a rating region table (RRT)including information on a content advisory rating, SystemTimeinformation for providing information associated with a system time, acommon alert protocol (CAP) message for providing information associatedwith emergency alert. In some embodiments, the other information may beincluded in the LLS table.

One embodiment t3020 of the shown SLT may include an @bsid attribute, an@sltCapabilities attribute, an sltInetUrl element and/or a Serviceelement. Each field may be omitted according to the value of the shownUse column or a plurality of fields may be present.

The @bsid attribute may be the identifier of a broadcast stream. The@sltCapabilities attribute may provide capability information requiredto decode and significantly reproduce all services described in the SLT.The sltInetUrl element may provide base URL information used to obtainservice signaling information and ESG for the services of the SLT overbroadband. The sltInetUrl element may further include an @urlTypeattribute, which may indicate the type of data capable of being obtainedthrough the URL.

The Service element may include information on services described in theSLT, and the Service element of each service may be present. The Serviceelement may include an @serviceId attribute, an @sltSvcSeqNum attribute,an @protected attribute, an @majorChannelNo attribute, an@minorChannelNo attribute, an @serviceCategory attribute, an@shortServiceName attribute, an @hidden attribute, an@broadbandAccessRequired attribute, an @svcCapabilities attribute, aBroadcastSvcSignaling element and/or an svcInetUrl element.

The @serviceId attribute is the identifier of the service and the@sltSvcSeqNum attribute may indicate the sequence number of the SLTinformation of the service. The @protected attribute may indicatewhether at least one service component necessary for significantreproduction of the service is protected. The @majorChannelNo attributeand the @minorChannelNo attribute may indicate the major channel numberand minor channel number of the service, respectively.

The @serviceCategory attribute may indicate the category of the service.The category of the service may include a linear A/V service, a linearaudio service, an app based service, an ESG service, an EAS service,etc. The @shortServiceName attribute may provide the short name of theservice. The @hidden attribute may indicate whether the service is fortesting or proprietary use. The @broadbandAccessRequired attribute mayindicate whether broadband access is necessary for significantreproduction of the service. The @svcCapabilities attribute may providecapability information necessary for decoding and significantreproduction of the service.

The BroadcastSvcSignaling element may provide information associatedwith broadcast signaling of the service. This element may provideinformation such as location, protocol and address with respect tosignaling over the broadcast network of the service. Details thereofwill be described below.

The svcInetUrl element may provide URL information for accessing thesignaling information of the service over broadband. The sltInetUrlelement may further include an @urlType attribute, which may indicatethe type of data capable of being obtained through the URL.

The above-described BroadcastSvcSignaling element may include an@slsProtocol attribute, an @slsMajorProtocolVersion attribute, an@slsMinorProtocolVersion attribute, an @slsPlpId attribute, an@slsDestinationIpAddress attribute, an @slsDestinationUdpPort attributeand/or an @slsSourceIpAddress attribute.

The @slsProtocol attribute may indicate the protocol used to deliver theSLS of the service (ROUTE, MMT, etc.). The @slsMajorProtocolVersionattribute and the @slsMinorProtocolVersion attribute may indicate themajor version number and minor version number of the protocol used todeliver the SLS of the service, respectively.

The @slsPlpId attribute may provide a PLP identifier for identifying thePLP delivering the SLS of the service. In some embodiments, this fieldmay be omitted and the PLP information delivered by the SLS may bechecked using a combination of the information of the below-describedLMT and the bootstrap information of the SLT.

The @slsDestinationIpAddress attribute, the @slsDestinationUdpPortattribute and the @slsSourceIpAddress attribute may indicate thedestination IP address, destination UDP port and source IP address ofthe transport packets delivering the SLS of the service, respectively.These may identify the transport session (ROUTE session or MMTP session)delivered by the SLS. These may be included in the bootstrapinformation.

FIG. 4 is a diagram showing a USBD and an S-TSID delivered through ROUTEaccording to one embodiment of the present invention.

One embodiment t4010 of the shown USBD may have a bundleDescription rootelement. The bundleDescription root element may have auserServiceDescription element. The userServiceDescription element maybe an instance of one service.

The userServiceDescription element may include an @globalServiceIDattribute, an @serviceId attribute, an @serviceStatus attribute, an@fullMPDUri attribute, an @sTSIDUri attribute, a name element, aserviceLanguage element, a capabilityCode element and/or adeliveryMethod element. Each field may be omitted according to the valueof the shown Use column or a plurality of fields may be present.

The @globalServiceID attribute is the globally unique identifier of theservice and may be used for link with ESG data(Service@globalServiceID). The @serviceId attribute is a referencecorresponding to the service entry of the SLT and may be equal to theservice ID information of the SLT. The @serviceStatus attribute mayindicate the status of the service. This field may indicate whether theservice is active or inactive.

The @fullMPDUri attribute may reference the MPD fragment of the service.The MPD may provide a reproduction description of a service componentdelivered over the broadcast or broadband network as described above.The @sTSIDUri attribute may reference the S-TSID fragment of theservice. The S-TSID may provide parameters associated with access to thetransport session carrying the service as described above.

The name element may provide the name of the service. This element mayfurther include an @lang attribute and this field may indicate thelanguage of the name provided by the name element. The serviceLanguageelement may indicate available languages of the service. That is, thiselement may arrange the languages capable of being provided by theservice.

The capabilityCode element may indicate capability or capability groupinformation of a receiver necessary to significantly reproduce theservice. This information is compatible with capability informationformat provided in service announcement.

The deliveryMethod element may provide transmission related informationwith respect to content accessed over the broadcast or broadband networkof the service. The deliveryMethod element may include abroadcastAppService element and/or a unicastAppService element. Each ofthese elements may have a basePattern element as a sub element.

The broadcastAppService element may include transmission associatedinformation of the DASH representation delivered over the broadcastnetwork. The DASH representation may include media components over allperiods of the service presentation.

The basePattern element of this element may indicate a character patternused for the receiver to perform matching with the segment URL. This maybe used for a DASH client to request the segments of the representation.Matching may imply delivery of the media segment over the broadcastnetwork.

The unicastAppService element may include transmission relatedinformation of the DASH representation delivered over broadband. TheDASH representation may include media components over all periods of theservice media presentation.

The basePattern element of this element may indicate a character patternused for the receiver to perform matching with the segment URL. This maybe used for a DASH client to request the segments of the representation.Matching may imply delivery of the media segment over broadband.

One embodiment t4020 of the shown S-TSID may have an S-TSID rootelement. The S-TSID root element may include an @serviceId attributeand/or an RS element. Each field may be omitted according to the valueof the shown Use column or a plurality of fields may be present.

The @serviceId attribute is the identifier of the service and mayreference the service of the USBD/USD. The RS element may describeinformation on ROUTE sessions through which the service components ofthe service are delivered. According to the number of ROUTE sessions, aplurality of elements may be present. The RS element may further includean @bsid attribute, an @sIpAddr attribute, an @dIpAddr attribute, an@dport attribute, an @PLPID attribute and/or an LS element.

The @bsid attribute may be the identifier of a broadcast stream in whichthe service components of the service are delivered. If this field isomitted, a default broadcast stream may be a broadcast stream includingthe PLP delivering the SLS of the service. The value of this field maybe equal to that of the @bsid attribute.

The @sIpAddr attribute, the @dIpAddr attribute and the @dport attributemay indicate the source IP address, destination IP address anddestination UDP port of the ROUTE session, respectively. When thesefields are omitted, the default values may be the source address,destination IP address and destination UDP port values of the currentROUTE session delivering the SLS, that is, the S-TSID. This field maynot be omitted in another ROUTE session delivering the servicecomponents of the service, not in the current ROUTE session.

The @PLPID attribute may indicate the PLP ID information of the ROUTEsession. If this field is omitted, the default value may be the PLP IDvalue of the current PLP delivered by the S-TSID. In some embodiments,this field is omitted and the PLP ID information of the ROUTE sessionmay be checked using a combination of the information of thebelow-described LMT and the IP address/UDP port information of the RSelement.

The LS element may describe information on LCT channels through whichthe service components of the service are transmitted. According to thenumber of LCT channel, a plurality of elements may be present. The LSelement may include an @tsi attribute, an @PLPID attribute, an @bwattribute, an @startTime attribute, an @endTime attribute, a SrcFlowelement and/or a RepairFlow element.

The @tsi attribute may indicate the tsi information of the LCT channel.Using this, the LCT channels through which the service components of theservice are delivered may be identified. The @PLPID attribute mayindicate the PLP ID information of the LCT channel. In some embodiments,this field may be omitted. The @bw attribute may indicate the maximumbandwidth of the LCT channel. The @startTime attribute may indicate thestart time of the LCT session and the @endTime attribute may indicatethe end time of the LCT channel.

The SrcFlow element may describe the source flow of ROUTE. The sourceprotocol of ROUTE is used to transmit a delivery object and at least onesource flow may be established within one ROUTE session. The source flowmay deliver associated objects as an object flow.

The RepairFlow element may describe the repair flow of ROUTE. Deliveryobjects delivered according to the source protocol may be protectedaccording to forward error correction (FEC) and the repair protocol maydefine an FEC framework enabling FEC protection.

FIG. 5 is a diagram showing a USBD delivered through MMT according toone embodiment of the present invention.

One embodiment of the shown USBD may have a bundleDescription rootelement. The bundleDescription root element may have auserServiceDescription element. The userServiceDescription element maybe an instance of one service.

The userServiceDescription element may include an @globalServiceIDattribute, an @serviceId attribute, a Name element, a serviceLanguageelement, a contentAdvisoryRating element, a Channel element, ampuComponent element, a routeComponent element, a broadbandComponentelement and/or a ComponentInfo element. Each field may be omittedaccording to the value of the shown Use column or a plurality of fieldsmay be present.

The @globalServiceID attribute, the @serviceId attribute, the Nameelement and/or the serviceLanguage element may be equal to the fields ofthe USBD delivered through ROUTE. The contentAdvisoryRating element mayindicate the content advisory rating of the service. This information iscompatible with content advisory rating information format provided inservice announcement. The Channel element may include informationassociated with the service. A detailed description of this element willbe given below.

The mpuComponent element may provide a description of service componentsdelivered as the MPU of the service. This element may further include an@mmtPackageId attribute and/or an @nextMmtPackageId attribute. The@mmtPackageId attribute may reference the MMT package of the servicecomponents delivered as the MPU of the service. The @nextMmtPackageIdattribute may reference an MMT package to be used after the MMT packagereferenced by the @mmtPackageId attribute in terms of time. Through theinformation of this element, the MP table may be referenced.

The routeComponent element may include a description of the servicecomponents of the service. Even when linear service components aredelivered through the MMT protocol, NRT data may be delivered accordingto the ROUTE protocol as described above. This element may describeinformation on such NRT data. A detailed description of this elementwill be given below.

The broadbandComponent element may include the description of theservice components of the service delivered over broadband. In hybridservice delivery, some service components of one service or other filesmay be delivered over broadband. This element may describe informationon such data. This element may further an @fullMPDUri attribute. Thisattribute may reference the MPD describing the service componentdelivered over broadband. In addition to hybrid service delivery, thebroadcast signal may be weakened due to traveling in a tunnel and thusthis element may be necessary to support handoff between broadband andbroadband. When the broadcast signal is weak, the service component isacquired over broadband and, when the broadcast signal becomes strong,the service component is acquired over the broadcast network to secureservice continuity.

The ComponentInfo element may include information on the servicecomponents of the service. According to the number of service componentsof the service, a plurality of elements may be present. This element maydescribe the type, role, name, identifier or protection of each servicecomponent. Detailed information of this element will be described below.

The above-described Channel element may further include an @serviceGenreattribute, an @serviceIcon attribute and/or a ServiceDescriptionelement. The @serviceGenre attribute may indicate the genre of theservice and the @serviceIcon attribute may include the URL informationof the representative icon of the service. The ServiceDescriptionelement may provide the service description of the service and thiselement may further include an @serviceDescrText attribute and/or an@serviceDescrLang attribute. These attributes may indicate the text ofthe service description and the language used in the text.

The above-described routeComponent element may further include an@sTSIDUri attribute, an @sTSIDDestinationIpAddress attribute, an@sTSIDDestinationUdpPort attribute, an @sTSIDSourceIpAddress attribute,an @sTSIDMajorProtocolVersion attribute and/or an@sTSIDMinorProtocolVersion attribute.

The @sTSIDUri attribute may reference an S-TSID fragment. This field maybe equal to the field of the USBD delivered through ROUTE. This S-TSIDmay provide access related information of the service componentsdelivered through ROUTE. This S-TSID may be present for NRT datadelivered according to the ROUTE protocol in a state of deliveringlinear service component according to the MMT protocol.

The @sTSIDDestinationIpAddress attribute, the @sTSIDDestinationUdpPortattribute and the @sTSIDSourceIpAddress attribute may indicate thedestination IP address, destination UDP port and source IP address ofthe transport packets carrying the above-described S-TSID. That is,these fields may identify the transport session (MMTP session or theROUTE session) carrying the above-described S-TSID.

The @sTSIDMajorProtocolVersion attribute and the@sTSIDMinorProtocolVersion attribute may indicate the major versionnumber and minor version number of the transport protocol used todeliver the above-described S-TSID, respectively.

The above-described ComponentInfo element may further include an@componentType attribute, an @componentRole attribute, an@componentProtectedFlag attribute, an @componentId attribute and/or an@componentName attribute.

The @componentType attribute may indicate the type of the component. Forexample, this attribute may indicate whether the component is an audio,video or closed caption component. The @componentRole attribute mayindicate the role of the component. For example, this attribute mayindicate main audio, music, commentary, etc. if the component is anaudio component. This attribute may indicate primary video if thecomponent is a video component. This attribute may indicate a normalcaption or an easy reader type if the component is a closed captioncomponent.

The @componentProtectedFlag attribute may indicate whether the servicecomponent is protected, for example, encrypted. The @componentIdattribute may indicate the identifier of the service component. Thevalue of this attribute may be the asset_id (asset ID) of the MP tablecorresponding to this service component. The @componentName attributemay indicate the name of the service component.

FIG. 6 is a diagram showing link layer operation according to oneembodiment of the present invention.

The link layer may be a layer between a physical layer and a networklayer. A transmission side may transmit data from the network layer tothe physical layer and a reception side may transmit data from thephysical layer to the network layer (t6010). The purpose of the linklayer is to compress (abstract) all input packet types into one formatfor processing by the physical layer and to secure flexibility andexpandability of an input packet type which is not defined yet. Inaddition, the link layer may provide option for compressing(abstracting) unnecessary information of the header of input packets toefficiently transmit input data. Operation such as overhead reduction,encapsulation, etc. of the link layer is referred to as a link layerprotocol and packets generated using this protocol may be referred to aslink layer packets. The link layer may perform functions such as packetencapsulation, overhead reduction and/or signaling transmission.

At the transmission side, the link layer (ALP) may perform an overheadreduction procedure with respect to input packets and then encapsulatethe input packets into link layer packets. In addition, in someembodiments, the link layer may perform encapsulation into the linklayer packets without performing the overhead reduction procedure. Dueto use of the link layer protocol, data transmission overhead on thephysical layer may be significantly reduced and the link layer protocolaccording to the present invention may provide IP overhead reductionand/or MPEG-2 TS overhead reduction.

When the shown IP packets are input as input packets (t6010), the linklayer may sequentially perform IP header compression, adaptation and/orencapsulation. In some embodiments, some processes may be omitted. Forexample, the RoHC module may perform IP packet header compression toreduce unnecessary overhead. Context information may be extractedthrough the adaptation procedure and transmitted out of band. The IPheader compression and adaption procedure may be collectively referredto as IP header compression. Thereafter, the IP packets may beencapsulated into link layer packets through the encapsulationprocedure.

When MPEG 2 TS packets are input as input packets, the link layer maysequentially perform overhead reduction and/or an encapsulationprocedure with respect to the TS packets. In some embodiments, someprocedures may be omitted. In overhead reduction, the link layer mayprovide sync byte removal, null packet deletion and/or common headerremoval (compression). Through sync byte removal, overhead reduction of1 byte may be provided per TS packet. Null packet deletion may beperformed in a manner in which reinsertion is possible at the receptionside. In addition, deletion (compression) may be performed in a mannerin which common information between consecutive headers may be restoredat the reception side. Some of the overhead reduction procedures may beomitted. Thereafter, through the encapsulation procedure, the TS packetsmay be encapsulated into link layer packets. The link layer packetstructure for encapsulation of the TS packets may be different from thatof the other types of packets.

First, IP header compression will be described.

The IP packets may have a fixed header format but some informationnecessary for a communication environment may be unnecessary for abroadcast environment. The link layer protocol may compress the headerof the IP packet to provide a mechanism for reducing broadcast overhead.

IP header compression may include a header compressor/decompressorand/or an adaptation module. The IP header compressor (RoHC compressor)may reduce the size of each IP packet based on a RoHC method. Then,adaptation module may extract context information and generate signalinginformation from each packet stream. A receiver may parse signalinginformation related to a corresponding packet stream and attach thecontext information to the packet stream. The RoHC decompressor mayrecover a packet header to reconfigure an original IP packet.Hereinafter, IP header compression may refer to only IP headercompressor via header compressor and may be a concept that combines IPheader compression and the adaptation procedure by the adaptationmodule. This may be the same as in decompressing.

Hereinafter, adaptation will be described.

In transmission of a single-direction link, when the receiver does nothave context information, the decompressor cannot restore the receivedpacket header until complete context is received. This may lead tochannel change delay and turn-on delay. Accordingly, through theadaptation function, configuration parameters and context informationbetween the compressor and the decompressor may be transmitted out ofband. The adaptation function may construct link layer signaling usingcontext information and/or configuration parameters. The adaptationfunction may periodically transmit link layer signaling through eachphysical frame using a previous configuration parameter and/or contextinformation.

Context information is extracted from the compressed IP packets andvarious methods may be used according to adaptation mode.

Mode #1 refers to a mode in which no operation is performed with respectto the compressed packet stream and an adaptation module operates as abuffer.

Mode #2 refers to a mode in which an IR packet is detected from acompressed packet stream to extract context information (static chain).After extraction, the IR packet is converted into an IR-DYN packet andthe IR-DYN packet may be transmitted in the same order within the packetstream in place of an original IR packet.

Mode #3 (t6020) refers to a mode in which IR and IR-DYN packets aredetected from a compressed packet stream to extract context information.A static chain and a dynamic chain may be extracted from the IR packetand a dynamic chain may be extracted from the IR-DYN packet. Afterextraction, the IR and IR-DYN packets are converted into normalcompression packets. The converted packets may be transmitted in thesame order within the packet stream in place of original IR and IR-DYNpackets.

In each mode, the context information is extracted and the remainingpackets may be encapsulated and transmitted according to the link layerpacket structure for the compressed IP packets. The context informationmay be encapsulated and transmitted according to the link layer packetstructure for signaling information, as link layer signaling.

The extracted context information may be included in a RoHC-Udescription table (RDT) and may be transmitted separately from the RoHCpacket flow. Context information may be transmitted through a specificphysical data path along with other signaling information. The specificphysical data path may mean one of normal PLPs, a PLP in which low levelsignaling (LLS) is delivered, a dedicated PLP or an L1 signaling path.Here, the RDT may be context information (static chain and/or dynamicchain) and/or signaling information including information associatedwith header compression. In some embodiments, the RDT may be transmittedwhenever context information is changed. In some embodiments, the RDTmay be transmitted in every physical frame. To transmit the RDT in everyphysical frame, a previous RDT may be re-used.

The receiver may select a first PLP and first acquire signalinginformation of the SLT, the RDT, etc., prior to acquisition of a packetstream. Upon acquiring the signaling information, the receiver maycombine the information to acquire mapping of service—IPinformation—context information—PLP. That is, the receiver may recognizeIP streams through which a service is transmitted, IP streamstransmitted through a PLP, and so on and acquire corresponding contextinformation of the PLPs. The receiver may select a PLP for delivery of aspecific packet stream and decode the PLP. The adaptation module mayparse the context information and combine the context information withthe compressed packets. Thereby, the packet stream may be recovered andtransmitted to the RoHC de compressor. Then, decompression may bestarted. In this case, the receiver may detect an IR packet and startdecompression from a first received IR packet according to an adaptationmode (mode 1), may detect an IR-DYN packet and start decompression froma first received IR-DYN packet (mode 2), or may start decompression fromany general compressed packet (mode 3).

Hereinafter, packet encapsulation will be described.

The link layer protocol may encapsulate all types of input packets suchas IP packets, TS packets, etc. into link layer packets. To this end,the physical layer processes only one packet format independently of theprotocol type of the network layer (here, an MPEG-2 TS packet isconsidered as a network layer packet). Each network layer packet orinput packet is modified into the payload of a generic link layerpacket.

In the packet encapsulation procedure, segmentation may be used. If thenetwork layer packet is too large to be processed in the physical layer,the network layer packet may be segmented into two or more segments. Thelink layer packet header may include fields for segmentation of thetransmission side and recombination of the reception side. Each segmentmay be encapsulated into the link layer packet in the same order as theoriginal location.

In the packet encapsulation procedure, concatenation may also be used.If the network layer packet is sufficiently small such that the payloadof the link layer packet includes several network layer packets,concatenation may be performed. The link layer packet header may includefields for performing concatenation. In concatenation, the input packetsmay be encapsulated into the payload of the link layer packet in thesame order as the original input order.

The link layer packet may include a header and a payload. The header mayinclude a base header, an additional header and/or an optional header.The additional header may be further added according to situation suchas concatenation or segmentation and the additional header may includefields suitable for situations. In addition, for delivery of theadditional information, the optional header may be further included.Each header structure may be pre-defined. As described above, if theinput packets are TS packets, a link layer header having packetsdifferent from the other packets may be used.

Hereinafter, link layer signaling will be described.

Link layer signaling may operate at a level lower than that of the IPlayer. The reception side may acquire link layer signaling faster thanIP level signaling of the LLS, the SLT, the SLS, etc. Accordingly, linklayer signaling may be acquired before session establishment.

Link layer signaling may include internal link layer signaling andexternal link layer signaling. Internal link layer signaling may besignaling information generated at the link layer. This includes theabove-described RDT or the below-described LMT. External link layersignaling may be signaling information received from an external module,an external protocol or a higher layer. The link layer may encapsulatelink layer signaling into a link layer packet and deliver the link layerpacket. A link layer packet structure (header structure) for link layersignaling may be defined and link layer signaling information may beencapsulated according to this structure.

FIG. 7 is a diagram showing a link mapping table (LMT) according to oneembodiment of the present invention.

The LMT may provide a list of higher layer sessions carried through thePLP. In addition, the LMT may provide additional information forprocessing link layer packets carrying the higher layer sessions. Here,the higher layer session may be referred to as multicast. Information onIP streams or transport sessions transmitted through one PLP may beacquired through the LMT. In contrast, information on through which PLPa specific transport session is delivered may be acquired.

The LMT may be transmitted through any PLP identified to deliver theLLS. Here, the PLP for delivering the LLS may be identified by an LLSflag of L1 detail signaling information of a physical layer. The LLSflag may be a flag field indicating whether the LLS is transmittedthrough a corresponding PLP with respect to each PLP. Here, the L1detail signaling information may be correspond to PLS2 data which willbe described later.

That is, the LMT may also be transmitted through the same PLP along withthe LLS. Each LMT may describe mapping between PLPs and IP address/portas described above. As described above, the LLS may include an SLT and,in this regard, the IP address/ports described by the LMT may be any IPaddress/ports related to any service, described by the SLT transmittedthrough the PLP such as a corresponding LMT.

In some embodiments, the PLP identifier information in theabove-described SLT, SLS, etc. may be used to confirm informationindicating through which PLP a specific transport session indicated bythe SLT or SLS is transmitted may be confirmed.

In another embodiment, the PLP identifier information in theabove-described SLT, SLS, etc. will be omitted and PLP information ofthe specific transport session indicated by the SLT or SLS may beconfirmed by referring to the information in the LMT. In this case, thereceiver may combine the LMT and other IP level signaling information toidentify the PLP. Even in this embodiment, the PLP information in theSLT, SLS, etc. is not omitted and may remain in the SLT, SLS, etc.

The LMT according to the shown embodiment may include a signaling_typefield, a PLP_ID field, a num_session field and/or information on eachsession. Although the LMT of the shown embodiment describes IP streamstransmitted through one PLP, a PLP loop may be added to the LMT todescribe information on a plurality of PLPs in some embodiments. In thiscase, as described above, the LMT may describe PLPs of all IPaddresses/ports related to all service described by the SLT transmittedtogether using a PLP loop.

The signaling_type field may indicate the type of signaling informationdelivered by the table. The value of signaling_type field for the LMTmay be set to 0x01. The signaling_type field may signaling_type fieldmay be omitted. The PLP_ID field may identify a target PLP to bedescribed. When the PLP loop is used, each PLP_ID field may identifyeach target PLP. Fields from the PLP_ID field may be included in the PLPloop. Here, the below-described PLP_ID field may be an identifier of onePLP of the PLP loop and the following fields may be fields correspondingto the corresponding PLP.

The num_session field may indicate the number of higher layer sessionsdelivered through the PLP identified by the PLP_ID field. According tothe number indicated by the num_session field, information on eachsession may be included. This information may include a src_IP_addfield, a dst_IP_add field, a src_UDP_port field, a dst_UDP_port field,an SID_flag field, a compressed_flag field, an SID field, and/or acontext_id field.

The src_IP_add field, the dst_IP_add field, the src_UDP_port field, andthe dst_UDP_port field may indicate the source IP address, thedestination IP address, the source UDP port and the destination UDP portof the transport session among the higher layer sessions deliveredthrough the PLP identified by the PLP_ID field.

The SID_flag field may indicate whether the link layer packet deliveringthe transport session has an SID field in the optional header. The linklayer packet delivering the higher layer session may have an SID fieldin the optional header and the SID field value may be equal to that ofthe SID field in the LMT.

The compressed_flag field may indicate whether header compression isapplied to the data of the link layer packet delivering the transportsession. In addition, presence/absence of the below-described context_idfield may be determined according to the value of this field. Whenheader compression is applied (compressed_flag=1), the RDT may bepresent and the PLP ID field of the RDT may have the same value as thecorresponding PLP_ID field related to the present compressed_flag field.

The SID field may indicate a sub stream ID (SID) of link layer packetsfor delivering a corresponding transfer session. The link layer packetsmay include the SID having the same value as the present SID field inthe optional header. Thereby, the receiver may filter link layer packetsusing information of the LMT and SID information of a link layer packetheader without parsing of all link layer packets.

The context_id field may provide a reference for a context id (CID) inthe RDT. The CID information of the RDT may indicate the context ID ofthe compression IP packet stream. The RDT may provide contextinformation of the compression IP packet stream. Through this field, theRDT and the LMT may be associated.

In the above-described embodiments of the signaling information/table ofthe present invention, the fields, elements or attributes may be omittedor may be replaced with other fields. In some embodiments, additionalfields, elements or attributes may be added.

In one embodiment of the present invention, service components of oneservice may be delivered through a plurality of ROUTE sessions. In thiscase, an SLS may be acquired through bootstrap information of an SLT. AnS-TSID and an MPD may be referenced through the USBD of the SLS. TheS-TSID may describe not only the ROUTE session delivered by the SLS butalso transport session description information of another ROUTE sessioncarried by the service components. To this end, the service componentsdelivered through the plurality of ROUTE sessions may all be collected.This is similarly applicable to the case in which the service componentsof one service are delivered through a plurality of MMTP sessions. Forreference, one service component may be simultaneously used by theplurality of services.

In another embodiment of the present invention, bootstrapping of an ESGservice may be performed by a broadcast or broadband network. Byacquiring the ESG over broadband, URL information of the SLT may beused. ESG information may be requested using this URL.

In another embodiment of the present invention, one service component ofone service may be delivered over the broadcast network and the otherservice component may be delivered over broadband (hybrid). The S-TSIDmay describe components delivered over the broadcast network such thatthe ROUTE client acquires desired service components. In addition, theUSBD may have base pattern information to describe which segments (whichcomponents) are delivered through which path. Accordingly, the receivercan confirm a segment to be requested from the broadband service and asegment to be detected in a broadcast stream.

In another embodiment of the present invention, scalable coding of aservice may be performed. The USBD may have all capability informationnecessary to render the service. For example, when one service isprovided in HD or UHD, the capability information of the USBD may have avalue of “HD or UHD”. The receiver may check which component isreproduced in order to render the UHD or HD service using the MPD.

In another embodiment of the present invention, through a TOI field ofthe LCT packets delivered through the LCT channel delivering the SLS,which SLS fragment is delivered using the LCT packets (USBD, S-TSID,MPD, etc.) may be identified.

In another embodiment of the present invention, app components to beused for app based enhancement/an app based service may be deliveredover the broadcast network as NRT components or may be delivered overbroadband. In addition, app signaling for app based enhancement may beperformed by an application signaling table (AST) delivered along withthe SLS. In addition, an event which is signaling for operation to beperformed by the app may be delivered in the form of an event messagetable (EMT) along with the SLS, may be signaled in the MPD or may bein-band signaled in the form of a box within DASH representation. TheAST, the EMT, etc. may be delivered over broadband. App basedenhancement, etc. may be provided using the collected app components andsuch signaling information.

In another embodiment of the present invention, a CAP message may beincluded and provided in the above-described LLS table for emergencyalert. Rich media content for emergency alert may also be provided. Richmedia may be signaled by a CAP message and, if rich media is present,the rich media may be provided as an EAS service signaled by the SLT.

In another embodiment of the present invention, linear servicecomponents may be delivered over the broadcast network according to theMMT protocol. In this case, NRT data (e.g., app components) of theservice may be delivered over the broadcast network according to theROUTE protocol. In addition, the data of the service may be deliveredover broadband. The receiver may access the MMTP session delivering theSLS using the bootstrap information of the SLT. The USBD of the SLSaccording to the MMT may reference the MP table such that the receiveracquires linear service components formatted into the MPU deliveredaccording to the MMT protocol. In addition, the USBD may furtherreference the S-TSID such that the receiver acquires NRT data deliveredaccording to the ROUTE protocol. In addition, the USBD may furtherreference the MPD to provide a reproduction description of datadelivered over broadband.

In another embodiment of the present invention, the receiver may deliverlocation URL information capable of acquiring a file content item (file,etc.) and/or a streaming component to a companion device through a websocket method. The application of the companion device may acquirecomponents, data, etc. through a request through HTTP GET using thisURL. In addition, the receiver may deliver information such as systemtime information, emergency alert information, etc. to the companiondevice.

FIG. 8 is a diagram showing a structure of a broadcast signaltransmission device of a next-generation broadcast service according toan embodiment of the present invention.

The broadcast signal transmission device of the next-generationbroadcast service according to an embodiment of the present inventionmay include an input format block 1000, a bit interleaved coding &modulation (BICM) block 1010, a frame building block 1020, an orthogonalfrequency division multiplexing (OFDM) generation block 1030, and asignaling generation block 1040. An operation of each block of thebroadcast signal transmission device will be described.

According to an embodiment of the present invention, input data may useIP stream/packet and MPEG2-TS as main input format and other streamtypes may be handled as a general stream.

The input format block 1000 may demultiplex each input stream using oneor more data pipes to which independent coding and modulation areapplied. The data pipe may be a basic unit for robustness control andmay affect quality of service (QoS). One or more services or servicecomponents may affect one data pipe. The data pipe may be a logicalchannel in a physical layer for delivering service data or metadata fordelivering one or more services or service components.

Since QoS is dependent upon the characteristics of a service provided bythe broadcast signal transmission device of the next-generationbroadcast service according to an embodiment of the present invention,data corresponding to each service needs to be processed via differentmethods.

The BICM block 1010 may include a processing block applied to a profile(or system) to which MIMO is not applied and/or a processing block of aprofile (or system) to which MIMO is applied and may include a pluralityof processing blocks for processing each data pipe.

The processing block of the BICM block to which MIMO is not applied mayinclude a data FEC encoder, a bit interleaver, a constellation mapper, asignal space diversity (SSD) encoding block, and a time interleaver. Theprocessing block of the BICM block to which MIMO is applied is differentfrom the processing block of the BICM to which MIMO is not applied inthat a cell word demultiplexer and an MIMO encoding block are furtherincluded.

The data FEC encoder may perform FEC encoding on an input BBF togenerate a FECBLOCK procedure using external coding (BCH) and internalcoding (LDPC). The external coding (BCH) may be a selective codingmethod. The bit interleaver may interleave output of the data FECencoder to achieve optimized performance using a combination of the LDPCcode and a modulation method. The constellation mapper may modulate cellword from a bit interleaver or a cell word demultiplexer using QPSK,QAM-16, irregular QAM (NUQ-64, NUQ-256, NUQ-1024), or irregularconstellation (NUC-16, NUC-64, NUC-256, NUC-1024) and provide apower-normalized constellation point. NUQ has an arbitrary type butQAM-16 and NUQ have a square shape. All of the NUQ and the NUC may beparticularly defined with respect to each code rate and signaled byparameter DP_MOD of PLS2 data. The time interleaver may be operated at adata pipe level. A parameter of the time interleaving may be differentlyset with respect to each data pipe.

The time interleaver according to the present invention may bepositioned between the BICM chain and the frame builder. In this case,the time interlever according to the present invention may selectivelyuse a convolution interleaver (CI) and a block interleaver (BI)according to a physical layer pipe (PLP) mode or may use all. The PLPaccording to an embodiment of the present invention may be a physicalpath used using the same concept as the aforementioned DP and its termmay be changed according to designer intention. The PLP mode accordingto an embodiment of the present invention may include a single PLP modeor a multiple PLP mode according to the number of PLPs processed by thebroadcast signal transmitter or the broadcast signal transmissiondevice. Time interleaving using different time interleaving methodsaccording to a PLP mode may be referred to as hybrid time interleaving.

A hybrid time interleaver may include a block interleaver (BI) and aconvolution interleaver (CI). In the case of PLP_NUM=1, the BI may notbe applied (BI off) and only the CI may be applied. In the case ofPLP_NUM>1, both the BI and the CI may be applied (BI on). The structureand operation of the CI applied in the case of PLP_NUM>1 may bedifferent from those of the CI applied in the case of PLP_NUM=1. Thehybrid time interleaver may perform an operation corresponding to areverse operation of the aforementioned hybrid time interleaver.

The cell word demultiplexer may be used to divide a single cell wordstream into a dual cell word stream for MIMO processing. The MIMOencoding block may process output of the cell word demultiplexer using aMIMO encoding method. The MIMO encoding method according to the presentinvention may be defined as full-rate spatial multiplexing (FR-SM) forproviding increase in capacity via relatively low increase in complexityat a receiver side. MIMO processing may be applied at a data pipe level.When a pair of constellation mapper outputs, NUQ e_(1,i) and e_(2,i) isinput to a MIMO encoder, a pair of MIMO encoder outputs, g1,i and g2,imay be transmitted by the same carrier k and OFDM symbol 1 of eachtransmission antenna.

The frame building block 1020 may map a data cell of an input data pipein one frame to an OFDM symbol and perform frequency interleaving forfrequency domain diversity.

According to an embodiment of the present invention, a frame may bedivided into a preamble, one or more frame signaling symbols (FSS), anda normal data symbol. The preamble may be a special symbol for providinga combination of basic transmission parameters for effectivetransmission and reception of a signal. The preamble may signal a basictransmission parameter and a transmission type of a frame. Inparticular, the preamble may indicate whether an emergency alert service(EAS) is currently provided in a current frame. The objective of the FSSmay be to transmit PLS data. For rapid synchronization and channelestimation and rapid decoding of PLS data, the FSS may have a pipepattern with higher density than a normal data symbol.

The frame building block may include a delay compensation block foradjusting timing between a data pipe and corresponding PLS data toensure co-time between a data pipe and corresponding PLS data at atransmitting side, a cell mapper for mapping a PLS, a data pipe, anauxiliary stream, a dummy stream, and so on to an active carrier of anOFDM symbol in a frame, and a frequency interleaver.

The frequency interleaver may randomly interleave a data cell receivedfrom the cell mapper to provide frequency diversity. The frequencyinterleaver may operate with respect to data corresponding to an OFDMsymbol pair including two sequential OFDM symbols or data correspondingto one OFDM symbol using different interleaving seed orders in order toacquire maximum interleaving gain in a single frame.

The OFDM generation block 1030 may modulate an OFDM carrier by the cellgenerated by the frame building block, insert a pilot, and generate atime domain signal for transmission. The corresponding block maysequentially insert guard intervals and may apply PAPR reductionprocessing to generate a last RF signal.

The signaling generation block 1040 may generate physical layersignaling information used in an operation of each functional block. Thesignaling information according to an embodiment of the presentinvention may include PLS data. The PLS may provide an element forconnecting a receiver to a physical layer data pipe. The PLS data mayinclude PLS1 data and PLS2 data.

The PLS1 data may be a first combination of PLS data transmitted to FSSin a frame with fixed size, coding, and modulation for transmittingbasic information on a system as well as a parameter required to dataPLS2 data. The PLS1 data may provide a basic transmission parameterincluding a parameter required to receive and decode PLS2 data. The PLS2data may be a second combination of PLP data transmitted to FSS fortransmitting more detailed PLS data of a data pipe and a system. PLS2signaling may further include two types of parameters of PLS2 staticdata (PLS2-STAT data) and PLS2 dynamic data (PLS2-DYN data). The PLS2static data may be PLS2 data that is static during duration of a framegroup and the PLS2 dynamic data may be PLS2 data that is dynamicallychanged every frame.

The PLS2 data may include FIC_FLAG information. A fast informationchannel (FIC) may be a dedicated channel for transmitting cross-layerinformation for enabling fast service acquisition and channel scanning.The FIC_FLAG information may indicate whether a fast information channel(FIC) is used in a current frame group via a 1-bit field. When a valueof the corresponding field is set to 1, the FIC may be provided in thecurrent frame. When a value of the corresponding field is set to 0, theFIC may not be transmitted in the current frame. The BICM block 1010 mayinclude a BICM block for protecting PLS data. The BICM block forprotecting the PLS data may include a PLS FEC encoder, a bitinterleaver, and a constellation mapper.

The PLS FEC encoder may include a scrambler for scrambling PLS1 data andPLS2 data, a BCH encoding/zero inserting block for performing externalencoding on the scrambled PLS 1 and 2 data using a BCH code shortenedfor PLS protection and inserting a zero bit after BCH encoding, a LDPCencoding block for performing encoding using an LDPC code, and an LDPCparity puncturing block. Only the PLS1 data may be permutated before anoutput bit of zero insertion is LDPC-encoded. The bit interleaver mayinterleave each of the shortened and punctured PLS1 data and PLS2 data,and the constellation mapper may map the bit-interleaved. PLS1 data andPLS2 data to constellation.

A broadcast signal reception device of a next-generation broadcastservice according to an embodiment of the present invention may performa reverse operation of the broadcast signal transmission device of thenext-generation broadcast service that has been described with referenceto FIG. 8.

The broadcast signal reception device of a next-generation broadcastservice according to an embodiment of the present invention may includea synchronization & demodulation module for performing demodulationcorresponding to a reverse operation performed by the broadcast signaltransmission device, a frame parsing module for parsing an input signalframe to extract data transmitted by a service selected by a user, ademapping & decoding module for converting an input signal into bitregion data, deinterleaving bit region data as necessary, performingdemapping on mapping applied for transmission efficiency, and correctingerror that occurs in a transmission channel for decoding, an outputprocessor for performing a reverse operation of variouscompression/signal processing procedures applied by the broadcast signaltransmission device, and a signaling decoding module for acquiring andprocessing PLS information from the signal demodulated by thesynchronization & demodulation module. The frame parsing module, thedemapping & decoding module, and the output processor may perform thefunctions using the PLS data output from the signaling decoding module.

Hereinafter, the timer interleaver will be described. A timeinterleaving group according to an embodiment of the present inventionmay be directly mapped to one frame or may be spread over P₁ frames. Inaddition, each time interleaving group may be divided into one or more(N_(TI)) time interleaving blocks. Here, each time interleaving blockmay correspond to one use of a time interleaver memory. A timeinterleaving block in the time interleaving group may include differentnumbers of XFECBLOCK. In general, the time interleaver may also functionas a buffer with respect to data pipe data prior to a frame generationprocedure.

The time interleaver according to an embodiment of the present inventionmay be a twisted row-column block interleaver. The twisted row-columnblock interleaver according to an embodiment of the present inventionmay write a first XFECBLOCK in a first column of the time interleavingmemory, write a second XFECBLOCK in a next column, and write theremaining XFECBLOCKs in the time interleaving block in the same manner.In an interleaving array, a cell may be read in a diagonal direction toa last row from a first row (a leftmost column as a start column is readalong a row in a right direction). In this case, to achieve singlememory deinterleaving at a receiver side irrespective of the number ofXFECBLOCK in the time interleaving block, the interleaving array for thetwisted row-column block interleaver may insert a virtual XFECBLOCK intothe time interleaving memory. In this case, to achieve single memorydeinterleaving at a receiver side, the virtual XFECBLOCK needs to beinserted into another frontmost XFECBLOCK.

FIG. 9 is a writing operation of a time interleaver according to anembodiment of the present invention.

A block shown in a left portion of the drawing shows a TI memory addressarray and a block shown in a right portion of the drawing shows awriting operation when two or one virtual FEC blocks are inserted into afrontmost group of TI groups with respect to two consecutive TI groups.

The frequency interleaver according to an embodiment of the presentinvention may include an interleaving address generator for generatingan interleaving address to be applied to data corresponding to a symbolpair.

FIG. 10 is a block diagram of an interleaving address generatorincluding a main-PRBS generator and a sub-PRBS generator according toeach FFT mode, included in the frequency interleaver, according to anembodiment of the present invention.

(a) is a block diagram of an interleaving address generator with respectto a 8K FFT mode. (b) is a block diagram of an interleaving addressgenerator with respect to a 16K FFT mode, and (c) is a block diagram ofan interleaving address generator with respect to a 32K FFT mode.

An interleaving procedure with respect to an OFDM symbol pair may useone interleaving sequence and will be described below. First, anavailable data cell (output cell from a cell mapper) to be interleavedin one OFDM symbol O_(m,1) may be defined as O_(m,1)=[x_(m,1,0), . . . ,x_(m,1,p), . . . , x_(m,1,Ndata−1)] with respect to l=0, . . . ,N_(sym)−1. In this case, x_(m,1,p) may be a p^(th) cell of a l^(th) OFDMsymbol in a m^(th) frame and N_(data) may be the number of data cells.In the case of a frame signaling symbol, N_(data)=C_(FSS), in the caseof normal data, N_(data)=C_(data), and in the case of a frame edgesymbol, N_(data)=C_(FES). In addition, the interleaving data cell may bedefined as P_(m,1)=[v_(m,1,0), . . . , v_(m,1,Ndata−1)] with respect tol=0, . . . , N_(sym)−1.

With respect to an OFDM symbol pair, an interleaved OFDM symbol pair maybe given according to v_(m,1,Hi(p))=x_(m,1,p), p=0, . . . , N_(data)−1for a first OFDM symbol of each pair and given according tov_(m,1,p)=x_(m,1,Hi(p)), p=0, . . . , N_(data)−1 for a second OFDMsymbol of each pair. In this case, H_(l)(p) may be an interleavingaddress generated based on a cyclic shift value (symbol offset) of aPRBS generator and a sub-PRBS generator.

FIG. 11 is a block diagram illustrating the network topology accordingto the embodiment.

As shown in FIG. 11, the network topology includes a content providingserver 10, a content recognizing service providing server 20, amulti-channel video distributing server 30, an enhanced serviceinformation providing server 40, a plurality of enhanced serviceproviding servers 50, a broadcast receiving device 60, a network 70, anda video display device 100.

The content providing server 10 may correspond to a broadcasting stationand broadcasts a broadcast signal including main audio-visual contents.The broadcast signal may further include enhanced services. The enhancedservices may or may not relate to main audio-visual contents. Theenhanced services may have formats such as service information,metadata, additional data, compiled execution files, web applications,Hypertext Markup Language (HTML) documents, XML documents, CascadingStyle Sheet (CSS) documents, audio files, video files, ATSC 2.0contents, and addresses such as Uniform Resource Locator (URL). Theremay be at least one content providing server.

The content recognizing service providing server 20 provides a contentrecognizing service that allows the video display device 100 torecognize content on the basis of main audio-visual content. The contentrecognizing service providing server 20 may or may not edit the mainaudio-visual content. There may be at least one content recognizingservice providing server.

The content recognizing service providing server 20 may be a watermarkserver that edits the main audio-visual content to insert a visiblewatermark, which may look a logo, into the main audio-visual content.This watermark server may insert the logo of a content provider at theupper-left or upper-right of each frame in the main audio-visual contentas a watermark.

Additionally, the content recognizing service providing server 20 may bea watermark server that edits the main audio-visual content to insertcontent information into the main audio-visual content as an invisiblewatermark.

Additionally, the content recognizing service providing server 20 may bea fingerprint server that extracts feature information from some framesor audio samples of the main audio-visual content and stores it. Thisfeature information is called signature.

The multi-channel video distributing server 30 receives and multiplexesbroadcast signals from a plurality of broadcasting stations and providesthe multiplexed broadcast signals to the broadcast receiving device 60.Especially, the multi-channel video distributing server 30 performsdemodulation and channel decoding on the received broadcast signals toextract main audio-visual content and enhanced service, and then,performs channel encoding on the extracted main audio-visual content andenhanced service to generate a multiplexed signal for distribution. Atthis point, since the multi channel video distributing server 30 mayexclude the extracted enhanced service or may add another enhancedservice, a broadcasting station may not provide services led by it.There may be at least one multi channel video distributing server.

The broadcasting device 60 may tune a channel selected by a user andreceives a signal of the tuned channel, and then, performs demodulationand channel decoding on the received signal to extract a mainaudio-visual content. The broadcasting device 60 decodes the extractedmain audio-visual content through H.264/Moving Picture Experts Group-4advanced video coding (MPEG-4 AVC), Dolby AC-3 or Moving Picture ExpertsGroup-2 Advanced Audio Coding (MPEG-2 AAC) algorithm to generate anuncompressed main audio-visual (AV) content. The broadcast receivingdevice 60 provides the generated uncompressed main AV content to thevideo display device 100 through its external input port.

The enhanced service information providing server 40 provides enhancedservice information on at least one available enhanced service relatingto a main AV content in response to a request of a video display device.There may be at least one enhanced service providing server. Theenhanced service information providing server 40 may provide enhancedservice information on the enhanced service having the highest priorityamong a plurality of available enhanced services.

The enhanced service providing server 50 provides at least one availableenhanced service relating to a main AV content in response to a requestof a video display device. There may be at least one enhanced serviceproviding server.

The video display device 100 may be a television, a notebook computer, ahand phone, and a smart phone, each including a display unit. The videodisplay device 100 may receive an uncompressed main AV content from thebroadcast receiving device 60 or a broadcast signal including an encodedmain AV content from the contents providing server 10 or the multichannel video distributing server 30. The video display device 100 mayreceive a content recognizing service from the content recognizingservice providing server 20 through the network 70, an address of atleast one available enhanced service relating to a main AV content fromthe enhanced service information providing server 40 through the network70, and at least one available enhanced service relating to a main AVcontent from the enhanced service providing server 50.

At least two of the content providing server 10, the content recognizingservice providing server 20, the multi channel video distributing server30, the enhanced service information providing server 40, and theplurality of enhanced service providing servers 50 may be combined in aform of one server and may be operated by one provider.

FIG. 12 is a block diagram illustrating a watermark based networktopology according to an embodiment.

As shown in FIG. 12, the watermark based network topology may furtherinclude a watermark server 21.

As shown in FIG. 12, the watermark server 21 edits a main AV content toinsert content information into it. The multi channel video distributingserver 30 may receive and distribute a broadcast signal including themodified main AV content. Especially, a watermark server may use adigital watermarking technique described below.

A digital watermark is a process for inserting information, which may bealmost undeletable, into a digital signal. For example, the digitalsignal may be audio, picture, or video. If the digital signal is copied,the inserted information is included in the copy. One digital signal maycarry several different watermarks simultaneously.

In visible watermarking, the inserted information may be identifiable ina picture or video. Typically, the inserted information may be a text orlogo identifying a media owner. If a television broadcasting stationadds its logo in a corner of a video, this is an identifiable watermark.

In invisible watermarking, although information as digital data is addedto audio, picture, or video, a user may be aware of a predeterminedamount of information but may not recognize it. A secret message may bedelivered through the invisible watermarking.

One application of the watermarking is a copyright protection system forpreventing the illegal copy of digital media. For example, a copy deviceobtains a watermark from digital media before copying the digital mediaand determines whether to copy or not on the bases of the content of thewatermark.

Another application of the watermarking is source tracking of digitalmedia. A watermark is embedded in the digital media at each point of adistribution path. If such digital media is found later, a watermark maybe extracted from the digital media and a distribution source may berecognized from the content of the watermark.

Another application of invisible watermarking is a description fordigital media.

A file format for digital media may include additional informationcalled metadata and a digital watermark is distinguished from metadatain that it is delivered as an AV signal itself of digital media.

The watermarking method may include spread spectrum, quantization, andamplitude modulation.

If a marked signal is obtained through additional editing, thewatermarking method corresponds to the spread spectrum. Although it isknown that the spread spectrum watermark is quite strong, not muchinformation is contained because the watermark interferes with anembedded host signal.

If a marked signal is obtained through the quantization, thewatermarking method corresponds to a quantization type. The quantizationwatermark is weak, much information may be contained.

If a marked signal is obtained through an additional editing methodsimilar to the spread spectrum in a spatial domain, a watermarkingmethod corresponds to the amplitude modulation.

FIG. 13 is a ladder diagram illustrating a data flow in a watermarkbased network topology according to an embodiment.

First, the content providing server 10 transmits a broadcast signalincluding a main AV content and an enhanced service in operation S101.

The watermark server 21 receives a broadcast signal that the contentproviding server 10 provides, inserts a visible watermark such as a logoor watermark information as an invisible watermark into the main AVcontent by editing the main AV content, and provides the watermarkedmain AV content and enhanced service to the MVPD 30 in operation S103.

The watermark information inserted through an invisible watermark mayinclude at least one of a watermark purpose, content information,enhanced service information, and an available enhanced service. Thewatermark purpose represents one of illegal copy prevention, viewerratings, and enhanced service acquisition.

The content information may include at least one of identificationinformation of a content provider that provides main AV content, main AVcontent identification information, time information of a contentsection used in content information acquisition, names of channelsthrough which main AV content is broadcasted, logos of channels throughwhich main AV content is broadcasted, descriptions of channels throughwhich main AV content is broadcasted, a usage information reportingperiod, the minimum usage time for usage information acquisition, andavailable enhanced service information relating to main AV content.

If the video display device 100 uses a watermark to acquire contentinformation, the time information of a content section used for contentinformation acquisition may be the time information of a content sectioninto which a watermark used is embedded. If the video display device 100uses a fingerprint to acquire content information, the time informationof a content section used for content information acquisition may be thetime information of a content section where feature information isextracted. The time information of a content section used for contentinformation acquisition may include at least one of the start time of acontent section used for content information acquisition, the durationof a content section used for content information acquisition, and theend time of a content section used for content information acquisition.

The usage information reporting address may include at least one of amain AV content watching information reporting address and an enhancedservice usage information reporting address. The usage informationreporting period may include at least one of a main AV content watchinginformation reporting period and an enhanced service usage informationreporting period. A minimum usage time for usage information acquisitionmay include at least one of a minimum watching time for a main AVcontent watching information acquisition and a minimum usage time forenhanced service usage information extraction.

On the basis that a main AV content is watched for more than the minimumwatching time, the video display device 100 acquires watchinginformation of the main AV content and reports the acquired watchinginformation to the main AV content watching information reportingaddress in the main AV content watching information reporting period.

On the basis that an enhanced service is used for more than the minimumusage time, the video display device 100 acquires enhanced service usageinformation and reports the acquired usage information to the enhancedservice usage information reporting address in the enhanced serviceusage information reporting period.

The enhanced service information may include at least one of informationon whether an enhanced service exists, an enhanced service addressproviding server address, an acquisition path of each available enhancedservice, an address for each available enhanced service, a start time ofeach available enhanced service, an end time of each available enhancedservice, a lifetime of each available enhanced service, an acquisitionmode of each available enhanced service, a request period of eachavailable enhanced service, priority information each available enhancedservice, description of each available enhanced service, a category ofeach available enhanced service, a usage information reporting address,a usage information reporting period, and the minimum usage time forusage information acquisition.

The acquisition path of available enhanced service may be representedwith IP or Advanced Television Systems Committee—Mobile/Handheld (ATSCM/H). If the acquisition path of available enhanced service is ATSC M/H,enhanced service information may further include frequency informationand channel information. An acquisition mode of each available enhancedservice may represent Push or Pull.

Moreover, the watermark server 21 may insert watermark information as aninvisible watermark into the logo of a main AV content.

For example, the watermark server 21 may insert a barcode at apredetermined position of a logo. At this point, the predeterminedposition of the logo may correspond to the first line at the bottom ofan area where the logo is displayed. The video display device 100 maynot display a barcode when receiving a main AV content including a logowith the barcode inserted.

For example, the watermark server 21 may insert a barcode at apredetermined position of a logo. At this point, the log may maintainits form.

For example, the watermark server 21 may insert N-bit watermarkinformation at each of the logos of M frames. That is, the watermarkserver 21 may insert M*N watermark information in M frames.

The MVPD 30 receives broadcast signals including watermarked main AVcontent and enhanced service and generates a multiplexed signal toprovide it to the broadcast receiving device 60 in operation S105. Atthis point, the multiplexed signal may exclude the received enhancedservice or may include new enhanced service.

The broadcast receiving device 60 tunes a channel that a user selectsand receives signals of the tuned channel, demodulates the receivedsignals, performs channel decoding and AV decoding on the demodulatedsignals to generate an uncompressed main AV content, and then, providesthe generated uncompressed main AV content to the video display device100 in operation S106.

Moreover, the content providing server 10 also broadcasts a broadcastsignal including a main AV content through a wireless channel inoperation S107.

Additionally, the MVPD 30 may directly transmit a broadcast signalincluding a main AV content to the video display device 100 withoutgoing through the broadcast receiving device 60 in operation S108.

The video display device 100 may receive an uncompressed main AV contentthrough the broadcast receiving device 60. Additionally, the videodisplay device 100 may receive a broadcast signal through a wirelesschannel, and then, may demodulate and decode the received broadcastsignal to obtain a main AV content. Additionally, the video displaydevice 100 may receive a broadcast signal from the MVPD 30, and then,may demodulate and decode the received broadcast signal to obtain a mainAV content. The video display device 100 extracts watermark informationfrom some frames or a section of audio samples of the obtained main AVcontent. If watermark information corresponds to a logo, the videodisplay device 100 confirms a watermark server address corresponding toa logo extracted from a corresponding relationship between a pluralityof logos and a plurality of watermark server addresses. When thewatermark information corresponds to the logo, the video display device100 cannot identify the main AV content only with the logo.Additionally, when the watermark information does not include contentinformation, the video display device 100 cannot identify the main AVcontent but the watermark information may include content provideridentifying information or a watermark server address. When thewatermark information includes the content provider identifyinginformation, the video display device 100 may confirm a watermark serveraddress corresponding to the content provider identifying informationextracted from a corresponding relationship between a plurality ofcontent provider identifying information and a plurality of watermarkserver addresses. In this manner, when the video display device 100cannot identify a main AV content the video display device 100 only withthe watermark information, it accesses the watermark server 21corresponding to the obtained watermark server address to transmit afirst query in operation S109.

The watermark server 21 provides a first reply to the first query inoperation S111. The first reply may include at least one of contentinformation, enhanced service information, and an available enhancedservice.

If the watermark information and the first reply do not include anenhanced service address, the video display device 100 cannot obtainenhanced service. However, the watermark information and the first replymay include an enhanced service address providing server address. Inthis manner, the video display device 100 does not obtain a serviceaddress or enhanced service through the watermark information and thefirst reply. If the video display device 100 obtains an enhanced serviceaddress providing server address, it accesses the enhanced serviceinformation providing server 40 corresponding to the obtained enhancedservice address providing server address to transmit a second queryincluding content information in operation S119.

The enhanced service information providing server 40 searches at leastone available enhanced service relating to the content information ofthe second query. Later, the enhanced service information providingserver 40 provides to the video display device 100 enhanced serviceinformation for at least one available enhanced service as a secondreply to the second query in operation S121.

If the video display device 100 obtains at least one available enhancedservice address through the watermark information, the first reply, orthe second reply, it accesses the at least one available enhancedservice address to request enhanced service in operation S123, and then,obtains the enhanced service in operation S125.

FIG. 14 is a block diagram illustrating a structure of a fingerprintbased video display device according to another embodiment.

As shown in FIG. 14, a tuner 501 extracts a symbol from an 8-VSB RFsignal transmitted through an air channel.

An 8-VSB demodulator 503 demodulates the 8-VSB symbol that the tuner 501extracts and restores meaningful digital data.

A VSB decoder 505 decodes the digital data that the 8-VSB demodulator503 to restore an ATSC main service and ATSC M/H service.

An MPEG-2 TP Demux 507 filters a Transport Packet that the video displaydevice 100 is to process from an MPEG-2 Transport Packet transmittedthrough an 8-VSB signal or an MPEG-2 Transport Packet stored in a PVRStorage to relay the filtered Transport Packet into a processing module.

A PES decoder 539 buffers and restores a Packetized Elementary Streamtransmitted through an MPEG-2 Transport Stream.

A PSI/PSIP decoder 541 buffers and analyzes PSI/PSIP Section Datatransmitted through an MPEG-2 Transport Stream. The analyzed PSI/PSIPdata are collected by a Service Manager (not shown), and then, is storedin DB in a form of Service Map and Guide data.

A DSMCC Section Buffer/Handler 511 buffers and processes DSMCC SectionData for file transmission through MPEG-2 TP and IP Datagramencapsulation.

An IP/UDP Datagram Buffer/Header Parser 513 buffers and restores IPDatagram, which is encapsulated through DSMCC Addressable section andtransmitted through MPEG-2 TP to analyze the Header of each Datagram.Additionally, an IP/UDP Datagram Buffer/Header Parser 513 buffers andrestores UDP Datagram transmitted through IP Datagram, and then analyzesand processes the restored UDP Header.

A Stream component handler 557 may include ES Buffer/Handler, PCRHandler, STC module, Descrambler, CA Stream Buffer/Handler, and ServiceSignaling Section Buffer/Handler.

The ES Buffer/Handler buffers and restores an Elementary Stream such asVideo and Audio data transmitted in a PES form to deliver it to a properA/V Decoder.

The PCR Handler processes Program Clock Reference (PCR) Data used forTime synchronization of Audio and Video Stream.

The STC module corrects Clock values of the A/V decoders by using aReference Clock value received through PCR Handler to perform TimeSynchronization.

When scrambling is applied to the received IP Datagram, the Descramblerrestores data of Payload by using Encryption key delivered from the CAStream Handler.

The CA Stream Buffer/Handler buffers and processes Data such as Keyvalues for Descrambling of EMM and ECM, which are transmitted for aConditional Access function through MPEG-2 TS or IP Stream. An output ofthe CA Stream Buffer/Handler is delivered to the Descrambler, and then,the descrambler descrambles MPEG-2 TP or IP Datagram, which carriers A/VData and File Data.

The Service Signaling Section Buffer/Handler buffers, restores, andanalyzes NRT Service Signaling Channel Section Data transmitted in aform of IP Datagram. The Service Manager (not shown) collects theanalyzed NRT Service Signaling Channel Section data and stores them inDB in a form of Service Map and Guide data.

The A/V Decoder 561 decodes the Audio/Video data received through an ESHandler to present them to a user.

An MPEG-2 Service Demux (not shown) may include an MPEG-2 TPBuffer/Parser, a Descrambler, and a PVR Storage module.

An MPEG-2 TP Buffer/Parser (not shown) buffers and restores an MPEG-2Transport Packet transmitted through an 8-VSB signal, and also detectsand processes a Transport Packet Header.

The Descrambler restores the data of Payload by using an Encryption key,which is delivered from the CA Stream Handler, on the Scramble appliedPacket payload in the MPEG-2 TP.

The PVR Storage module stores an MPEG-2 TP received through an 8-VSBsignal at the user's request and outputs an MPEG-2 TP at the user'srequest. The PVR storage module may be controlled by the PVR manager(not shown).

The File Handler 551 may include an ALC/LCT Buffer/Parser, an FDTHandler, an XML Parser, a File Reconstruction Buffer, a Decompressor, aFile Decoder, and a File Storage.

The ALC/LCT Buffer/Parser buffers and restores ALC/LCT data transmittedthrough a UDP/IP Stream, and analyzes a Header and Header extension ofALC/LCT. The ALC/LCT Buffer/Parser may be controlled by an NRT ServiceManager (not shown).

The FDT Handler analyzes and processes a File Description Table of FLUTEprotocol transmitted through an ALC/LCT session. The FDT Handler may becontrolled by an NRT Service Manager (not shown).

The XML Parser analyzes an XML Document transmitted through an ALC/LCTsession, and then, delivers the analyzed data to a proper module such asan FDT Handler and an SG Handler.

The File Reconstruction Buffer restores a file transmitted through anALC/LCT, FLUTE session.

If a file transmitted through an ALC/LCT and FLUTE session iscompressed, the Decompressor performs a process to decompress the file.

The File Decoder decodes a file restored in the File ReconstructionBuffer, a file decompressed in the decompressor, or a film extractedfrom the File Storage.

The File Storage stores or extracts a restored file if necessary.

The M/W Engine (not shown) processes data such as a file, which is notan A/V Stream transmitted through DSMCC Section and IP Datagram. The M/WEngine delivers the processed data to a Presentation Manager module.

The SG Handler (not shown) collects and analyzes Service Guide datatransmitted in an XML Document form, and then, delivers them to the EPGManager.

The Service Manager (not shown) collects and analyzes PSI/PSIP Datatransmitted through an MPEG-2 Transport Stream and Service SignalingSection Data transmitted through an IP Stream, so as to produce aService Map. The Service Manager (not shown) stores the produced servicemap in a Service Map & Guide Database, and controls an access to aService that a user wants. The Service Manager is controlled by theOperation Controller (not shown), and controls the Tuner 501, the MPEG-2TP Demux 507, and the IP Datagram Buffer/Handler 513.

The NRT Service Manager (not shown) performs an overall management onthe NRT service transmitted in an object/file form through a FLUTEsession. The NRT Service Manager (not shown) may control the FDT Handlerand File Storage.

The Application Manager (not shown) performs overall management onApplication data transmitted in a form of object and file.

The UI Manager (not shown) delivers a user input to an OperationController through a User Interface, and starts a process for a servicethat a user requests.

The Operation Controller (not shown) processes a command of a user,which is received through a UI Manager, and allows a Manager of anecessary module to perform a corresponding action.

The Fingerprint Extractor 565 extracts fingerprint feature informationfrom an AV stream.

The Fingerprint Comparator 567 compares the feature informationextracted by the Fingerprint Extractor with a Reference fingerprint tofind an identical content. The Fingerprint Comparator 567 may use aReference fingerprint DB stored in local and may query a Fingerprintquery server on the internet to receive a result. The matched resultdata obtained by a comparison result may be delivered to Application andused.

As an ACR function managing module or an application module providing anenhanced service on the basis of ACR, the Application 569 identifies abroadcast content in watching to provide an enhanced service related toit.

FIG. 15 is a block diagram illustrating a structure of a watermark basedvideo display device according to another embodiment of the presentinvention.

Although the watermark based video display device of FIG. 15 is similarto the fingerprint based video display device of FIG. 14, thefingerprint based video display device does not includes the FingerprintExtractor 565 and the Fingerprint Comparator 567, but further includesthe Watermark Extractor 566.

The Watermark Extractor 566 extracts data inserted in a watermark formfrom an Audio/Video stream. The extracted data may be delivered to anApplication and may be used.

FIG. 16 is a diagram showing data which may be delivered via awatermarking scheme according to one embodiment of the presentinvention.

As described above, an object of ACR via a WM is to obtain supplementaryservice related information of content from incompressible audio/videoin an environment capable of accessing only incompressible audio/video(that is, an environment in which audio/video is received from acable/satellite/IPTV, etc.). Such an environment may be referred to asan ACR environment. In the ACR environment, since a receiver receivesincompressible audio/video data only, the receiver may not confirm whichcontent is currently being displayed. Accordingly, the receiver uses acontent source ID, a current point of time of a broadcast program andURL information of a related application delivered by a WM to identifydisplayed content and provide an interactive service.

In delivery of a supplementary service related to a broadcast programusing an audio/video watermark (WM), all supplementary information maybe delivered by the WM as a simplest method. In this case, allsupplementary information may be detected by a WM detector tosimultaneously process information detected by the receiver.

However, in this case, if the amount of WMs inserted into audio/videodata increases, total quality of audio/video may deteriorate. For thisreason, only minimum necessary data may be inserted into the WM. Astructure of WM data for enabling a receiver to efficiently receive andprocess a large amount of information while inserting minimum data as aWM needs to be defined. A data structure used for the WM may be equallyused even in a fingerprinting scheme which is relatively less influencedby the amount of data.

As shown, data delivered via the watermarking scheme according to oneembodiment of the present invention may include an ID of a contentsource, a timestamp, an interactive application URL, a timestamp's type,a URL protocol type, an application event, a destination type, etc. Inaddition, various types of data may be delivered via the WM schemeaccording to the present invention.

The present invention proposes the structure of data included in a WMwhen ACR is performed via a WM scheme. For shown data types, a mostefficient structure is proposed by the present invention.

Data which can be delivered via the watermarking scheme according to oneembodiment of the present invention include the ID of the contentsource. In an environment using a set top box, a receiver (a terminal orTV) may not check a program name, channel information, etc. when amultichannel video programming distributor (MVPD) does not deliverprogram related information via the set top box. Accordingly, a uniqueID for identifying a specific content source may be necessary. In thepresent invention, an ID type of a content source is not limited.Examples of the ID of the content source may be as follows.

First, a global program ID may be a global identifier for identifyingeach broadcast program. This ID may be directly created by a contentprovider or may be created in the format specified by an authoritativebody. Examples of the ID may include TMSId of “TMS metadata” of NorthAmerica, an EIDR ID which is a movie/broadcast program identifier, etc.

A global channel ID may be a channel identifier for identifying allchannels. Channel numbers differ between MVPDs provided by a set topbox. In addition, even in the same MVPD, channel numbers may differaccording to services designated by users. The global channel ID may beused as a global identifier which is not influenced by an MVPD, etc.According to embodiments, a channel transmitted via a terrestrial wavemay be identified by a major channel number and a minor channel number.If only a program ID is used, since a problem may occur when severalbroadcast stations broadcast the same program, the global channel ID maybe used to specify a specific broadcast channel.

Examples of the ID of the content source to be inserted into a WM mayinclude a program ID and a channel ID. One or both of the program ID andthe channel ID or a new ID obtained by combining the two IDs may beinserted into the WM. According to embodiments, each ID or combined IDmay be hashed to reduce the amount of data. The ID of each contentsource may be of a string type or an integer type. In the case of theinteger type, the amount of transmitted data may be further reduced.

In addition, data which can be delivered via the watermarking schemeaccording to one embodiment of the present invention may include atimestamp. The receiver should know a point of time of currently viewedcontent. This time related information may be referred to as a timestampand may be inserted into the WM. The time related information may takethe form of an absolute time (UTC, GPS, etc.) or a media time. The timerelated information may be delivered up to a unit of milliseconds foraccuracy and may be delivered up to a smaller unit according toembodiments. The timestamp may have a variable length according to typeinformation of the timestamp.

Data which can be delivered via the watermarking scheme according to oneembodiment may include the URL of the interactive application. If aninteractive application related to a currently viewed broadcast programis present, the URL of the application may be inserted into the WM. Thereceiver may detect the WM, obtain the URL, and execute the applicationvia a browser.

FIG. 17 is a diagram showing the meanings of the values of the timestamptype field according to one embodiment of the present invention.

The present invention proposes a timestamp type field as one of datawhich can be delivered via a watermarking scheme. In addition, thepresent invention proposes an efficient data structure of a timestamptype field.

The timestamp type field may be allocated 5 bits. The first two bits ofthe timestamp may mean the size of the timestamp and the next 3 bits maymean the unit of time information indicated by the timestamp. Here, thefirst two bits may be referred to as a timestamp size field and the next3 bits may be referred to as a timestamp unit field.

As shown, according to the size of the timestamp and the unit value ofthe timestamp, a variable amount of real timestamp information may beinserted into the WM. Using such variability, a designer may select asize allocated to the timestamp and the unit thereof according to theaccuracy of the timestamp. If accuracy of the timestamp increases, it ispossible to provide an interactive service at an accurate time. However,system complexity increases as accuracy of the timestamp increases. Inconsideration of this tradeoff, the size allocated to the timestamp andthe unit thereof may be selected.

If the first two bits of the timestamp type field are 00, the timestampmay have a size of 1 byte. If the first two bits of the timestamp typefield are 01, 10 and 11, the size of the timestamp may be 2, 4 and 8bytes, respectively.

If the last three bits of the timestamp type field are 000, thetimestamp may have a unit of milliseconds. If the last three bits of thetimestamp type field are 001, 010 and 011, the timestamp may havesecond, minute and hour units, respectively. The last three bits of thetimestamp type field of 101 to 111 may be reserved for future use.

Here, if the last three bits of the timestamp type field are 100, aseparate time code may be used as a unit instead of a specific time unitsuch as millisecond or second. For example, a time code may be insertedinto the WM in the form of HH:MM:SS:FF which is a time code form ofSMPTE. Here, HH may be an hour unit, MM may be a minute unit and SS maybe a second unit. FF may be frame information. Frame information whichis not a time unit may be simultaneously delivered to provide aframe-accurate service. A real timestamp may have a form of HHMMSSFFexcluding colon in order to be inserted into the WM. In this case, atimestamp size value may have 11 (8 bytes) and a timestamp unit valuemay be 100. In the case of a variable unit, how the timestamp isinserted is not limited by the present invention.

For example, if timestamp type information has a value of 10 andtimestamp unit information has a value of 000, the size of the timestampmay be 4 bits and the unit of the timestamp may be milliseconds. At thistime, if the timestamp is Ts=3265087, 3 digits 087 located at the backof the timestamp may mean a unit of milliseconds and the remainingdigits 3265 may mean a second unit. Accordingly, when this timestamp isinterpreted, a current time may mean that 54 minutes 25.087 seconds haselapsed after the program, into which the WM is inserted, starts. Thisis only exemplary and the timestamp serves as a wall time and mayindicate a time of a receiver or a segment regardless of content.

FIG. 18 is a diagram showing meanings of values of a URL protocol typefield according to one embodiment of the present invention.

The present invention proposes a URL protocol type field as one of datawhich can be delivered via a watermarking scheme. In addition, thepresent invention proposes an efficient data structure of a URL protocoltype field.

Among the above-described information, the length of the URL isgenerally long such that the amount of data to be inserted is relativelylarge. As described above, as the amount of data to be inserted into theWM decreases, efficiency increases. Thus, a fixed portion of the URL maybe processed by the receiver. Accordingly, the present inventionproposes a URL protocol type field.

The URL protocol type field may have a size of 3 bits. A serviceprovider may set a URL protocol in a WM using the URL protocol typefield. In this case, the URL of the interactive application may beinserted starting from a domain and may be transmitted to the WM.

A WM detector of the receiver may first parse the URL protocol typefield, obtain URL protocol information and prefix the protocol to theURL value transmitted thereafter, thereby generating an entire URL. Thereceiver may access the completed URL via a browser and execute theinteractive application.

Here, if the value of the URL protocol type field is 000, the URLprotocol may be directly specified and inserted into the URL field ofthe WM. If the value of the URL protocol type field is 001, 010 and 011,the URL protocols may be http://, https:// and ws://, respectively. TheURL protocol type field values of 100 to 111 may be reserved for futureuse.

The application URL may enable execution of the application via thebrowser (in the form of a web application). In addition, according toembodiments, a content source ID and timestamp information should bereferred to. In the latter case, in order to deliver the content sourceID information and the timestamp information to a remote server, a finalURL may be expressed in the following form.

Request URL: http://domain/path?cid=1233456&t=5005

In this embodiment, a content source ID may be 123456 and a timestampmay be 5005. cid may mean a query identifier of a content source ID tobe reported to the remote server. t may mean a query identifier of acurrent time to be reported to the remote server.

FIG. 19 is a flowchart illustrating a process of processing a URLprotocol type field according to one embodiment of the presentinvention.

First, a service provider 47010 may deliver content to a WM inserter47020 (s47010). Here, the service provider 47010 may perform a functionsimilar to the above-described content provision server.

The WM inserter 47020 may insert a WM into the delivered content(s47020). Here, the WM inserter 47020 may perform a function similar tothe above-described watermark server. The WM inserter 47020 may insertthe above-described WM into audio or video by a WM algorithm. Here, theinserted WM may include the above-described application URL information,content source ID information, etc. For example, the inserted WM mayinclude the above-described timestamp type field, the timestamp, thecontent ID, etc. The above-described protocol type field may have avalue of 001 and URL information may have a value of atsc.org. Thevalues of the field inserted into the WM are only exemplary and thepresent invention is not limited to this embodiment.

The WM inserter 47020 may transmit content, into which the WM isinserted (s47030). Transmission of the content, into which the WM isinserted, may be performed by the service provider 47010.

An STB 47030 may receive the content, into which the WM is inserted, andoutput incompressible A/V data (or raw A/V data) (s47040). Here, the STB47030 may mean the above-described broadcast reception apparatus or theset top box. The STB 47030 may be mounted inside or outside thereceiver.

A WM detector 47040 may detect the inserted WM from the receivedincompressible A/V data (s47050). The WM detector 47040 may detect theWM inserted by the WM inserter 47020 and deliver the detected WM to a WMmanager.

The WM manager 47050 may parse the detected WM (s47060). In theabove-described embodiment, the WM may have a URL protocol type fieldvalue of 001 and a URL value of atsc.org. Since the URL protocol typefield value is 001, this may mean that http://protocol is used. The WMmanager 47050 may combine http:// and atsc.org using this information togenerate an entire URL (s47070).

The WM manager 47050 may send the completed URL to a browser 47060 andlaunch an application (s47080). In some cases, if the content source IDinformation and the timestamp information should also be delivered, theapplication may be launched in the form ofhttp://atsc.org?cid=xxx&t=YYY.

The WM detector 47040 and the WM manager 47050 of the terminal arecombined to perform the functions thereof in one module. In this case,steps s45050, s47060 and s47070 may be processed in one module.

FIG. 20 is a diagram showing the meanings of the values of an eventfield according to one embodiment of the present invention.

The present invention proposes an event field as one of the data whichcan be delivered via the watermarking scheme. In addition, the presentinvention proposes an efficient data structure of an event field.

The application may be launched via the URL extracted from the WM. Theapplication may be controlled via a more detailed event. Events whichcan control the application may be indicated and delivered by the eventfield. That is, if an interactive application related to a currentlyviewed broadcast program is present, the URL of the application may betransmitted and the application may be controlled using events.

The event field may have a size of 3 bits. If the value of the eventfield is 000, this may indicate a “Prepare” command. Prepare is apreparation step before executing the application. A receiver, which hasreceived this command, may download content items related to theapplication in advance. In addition, the receiver may release necessaryresources in order to execute the application. Here, releasing thenecessary resources may mean that a memory is cleaned or otherunfinished applications are finished.

If the event field value is 001, this may indicate an “Execute” command.Execute may be a command for executing the application. If the eventfield value is 010, this may indicate a “Suspend” command. Suspend maymean that the executed application is suspended. If the event fieldvalue is 011, this may indicate a “Kill” command. Kill may be a commandfor finishing the already executed application. The event field valuesof 100 to 111 may be reserved for future use.

FIG. 21 is a diagram showing the meanings of the values of a destinationtype field according to one embodiment of the present invention.

The present invention proposes a destination type field as one of datawhich can be delivered via a watermarking scheme. In addition, thepresent invention proposes an efficient data structure of a destinationtype field.

With development of DTV related technology, supplementary servicesrelated to broadcast content may be provided by a companion device aswell as a screen of a TV receiver. However, companion devices may notreceive broadcast programs or may receive broadcast programs but may notdetect a WM. Accordingly, among applications for providing asupplementary service related to currently broadcast content, if anapplication to be executed by a companion device is present, relatedinformation thereof should be delivered to the companion device.

At this time, even in an environment in which the receiver and thecompanion device interwork, it is necessary to know by which device anapplication or data detected from a WM is consumed. That is, informationabout whether the application or data is consumed by the receiver or thecompanion device may be necessary. In order to deliver such informationas the WM, the present invention proposes a destination type field.

The destination type field may have a size of 3 bits. If the value ofthe destination type field is 0x00, this may indicate that theapplication or data detected by the WM is targeted at all devices. Ifthe value of the destination type field is 0x01, this may indicate thatthe application or data detected by the WM is targeted at a TV receiver.If the value of the destination type field is 0x02, this may indicatethat the application or data detected by the WM is targeted at asmartphone. If the value of the destination type field is 0x03, this mayindicate that the application or data detected by the WM is targeted ata tablet. If the value of the destination type field is 0x04, this mayindicate that the application or data detected by the WM is targeted ata personal computer. If the value of the destination type field is 0x05,this may indicate that the application or data detected by the WM istargeted at a remote server. Destination type field values of 0x06 to0xFF may be reserved for future use.

Here, the remote server may mean a server having all supplementaryinformation related to a broadcast program. This remote server may belocated outside the terminal. If the remote server is used, the URLinserted into the WM may not indicate the URL of a specific applicationbut may indicate the URL of the remote server. The receiver maycommunicate with the remote server via the URL of the remote server andreceive supplementary information related to the broadcast program. Atthis time, the received supplementary information may be a variety ofinformation such as a genre, actor information, synopsis, etc. of acurrently broadcast program as well as the URL of an application relatedthereto. The received information may differ according to system.

According to another embodiment, each bit of the destination type fieldmay be allocated to each device to indicate the destination of theapplication. In this case, several destinations may be simultaneouslydesignated via bitwise OR.

For example, when 0x01 indicates a TV receiver, 0x02 indicates asmartphone, 0x04 indicates a tablet, 0x08 indicates a PC and 0x10indicates a remote server, if the destination type field has a value of0x6, the application or data may be targeted at the smartphone and thetablet.

According to the value of the destination type field of the WM parsed bythe above-described WM manager, the WM manager may deliver eachapplication or data to the companion device. In this case, the WMmanager is a module for processing interworking with the companiondevice in the receiver and may deliver information related to eachapplication or data.

FIG. 22 is a diagram showing the structure of data to be inserted into aWM according to embodiment #1 of the present invention.

In the present embodiment, data inserted into the WM may haveinformation such as a timestamp type field, a timestamp, a content ID,an event field, a destination type field, a URL protocol type field anda URL. Here, the order of data may be changed and each datum may beomitted according to embodiments.

In the present embodiment, a timestamp size field of the timestamp typefield may have a value of 01 and a timestamp unit field may have a valueof 000. This may mean that 2 bits are allocated to the timestamp and thetimestamp has a unit if milliseconds.

In addition, the event field has a value of 001, which means theapplication should be immediately executed. The destination type fieldhas a value of 0x02, which may mean that data delivered by the WM shouldbe delivered to the smartphone. Since the URL protocol type field has avalue of 001 and the URL has a value of atsc.org, this may mean that thesupplementary information or the URL of the application ishttp://atsc.org.

FIG. 23 is a flowchart illustrating a process of processing a datastructure to be inserted into a WM according to embodiment #1 of thepresent invention.

Step s51010 of, at the service provider, delivering content to the WMinserter, step s51020 of, at the WM inserter, inserting the receivedcontent into the WM, step s51030 of, at the WM inserter, transmittingthe content, into which the WM is inserted, step s51040 of, at the STB,receiving the content, into which the WM is inserted, and outputting theincompressible A/V data, step s51050 of, at the WM detector, detectingthe WM, step s51060, at the WM manager, parsing the detected WM and/orstep s51070 of, at the WM manager, generating an entire URL may be equalto the above-described steps.

The WM manager is a companion device protocol module in the receiveraccording to the destination type field of the parsed WM and may deliverrelated data (s51080). The companion device protocol module may manageinterworking and communication with the companion device in thereceiver. The companion device protocol module may be paired with thecompanion device. According to embodiments, the companion deviceprotocol module may be a UPnP device. According to embodiments, thecompanion device protocol module may be located outside the terminal.

The companion device protocol module may deliver the related data to thecompanion device according to the destination type field (s51090). Inembodiment #1, the value of the destination type field is 0x02 and thedata inserted into the WM may be data for a smartphone. Accordingly, thecompanion device protocol module may send the parsed data to thesmartphone. That is, in this embodiment, the companion device may be asmartphone.

According to embodiments, the WM manager or the device protocol modulemay perform a data processing procedure before delivering data to thecompanion device. The companion device may have portability but insteadmay have relatively inferior processing/computing capabilities and asmall amount of memory. Accordingly, the receiver may process datainstead of the companion device and deliver the processed data to thecompanion device.

Such processing may be implemented as various embodiments. First, the WMmanager or the companion device protocol module may select only datarequired by the companion device. In addition, according to embodiments,if the event field includes information indicating that the applicationis finished, the application related information may not be delivered.In addition, if data is divided and transmitted via several WMs, thedata may be stored and combined and then final information may bedelivered to the companion device. The receiver may performsynchronization using the timestamp instead of the companion device anddeliver a command related to the synchronized application or deliver analready synchronized interactive service to the companion device and thecompanion device may perform display only. Timestamp related informationmay not be delivered, a time base may be maintained in the receiver onlyand related information may be delivered to the companion device when acertain event is activated. In this case, the companion device mayactivate the event according to the time when the related information isreceived, without maintaining the time base.

Similarly to the above description, the WM detector and the WM managerof the terminal may be combined to perform the functions thereof in onemodule. In this case, steps s51050, s51060, s51070 and s51080 may beperformed in one module.

In addition, according to embodiments, the companion device may alsohave the WM detector. When each companion device receives a broadcastprogram, into which a WM is inserted, each companion device may directlydetect the WM and then deliver the WM to another companion device. Forexample, a smartphone may detect and parse a WM and deliver relatedinformation to a TV. In this case, the destination type field may have avalue of 0x01.

FIG. 24 is a diagram showing the structure of data to be inserted into aWM according to embodiment #2 of the present invention.

In the present embodiment, data inserted into the WM may haveinformation such as a timestamp type field, a timestamp, a content ID,an event field, a destination type field, a URL protocol type field anda URL. Here, the order of data may be changed and each datum may beomitted according to embodiments.

In the present embodiment, a timestamp size field of the timestamp typefield may have a value of 01 and a timestamp unit field may have a valueof 000. This may mean that 2 bits are allocated to the timestamp and thetimestamp has a unit of milliseconds. The content ID may have a value of123456.

In addition, the event field has a value of 001, which means theapplication should be immediately executed. The destination type fieldhas a value of 0x05, which may mean that data delivered by the WM shouldbe delivered to the remote server. Since the URL protocol type field hasa value of 001 and the URL has a value of remoteserver.com, this maymean that the supplementary information or the URL of the application ishttp://remoteserver.com.

As described above, if the remote server is used, supplementaryinformation of the broadcast program may be received from the remoteserver. At this time, the content ID and the timestamp may be insertedinto the URL of the remote server as parameters and requested from theremote server. According to embodiments, the remote server may obtaininformation about a currently broadcast program via support of API. Atthis time, the API may enable the remote server to acquire the contentID and the timestamp stored in the receiver or to deliver relatedsupplementary information.

In the present embodiment, if the content ID and the timestamp areinserted into the URL of the remote server as parameters, the entire URLmay be http://remoteserver.com?cid=1233456&t=5005. Here, cid may mean aquery identifier of a content source ID to be reported to the remoteserver. Here, t may mean a query identifier of a current time to bereported to the remote server.

FIG. 25 is a flowchart illustrating a process of processing a datastructure to be inserted into a WM according to embodiment #2 of thepresent invention.

Step s53010 of, at the service provider, delivering content to the WMinserter, step s53020 of, at the WM inserter, inserting the receivedcontent into the WM, step s53030 of, at the WM inserter, transmittingthe content, into which the WM is inserted, step s53040 of, at the STB,receiving the content, into which the WM is inserted, and outputting theincompressible A/V data, step s53050 of, at the WM detector, detectingthe WM, and step s53060, at the WM manager, parsing the detected WM maybe equal to the above-described steps.

The WM manager may communicate with the remote server via the parseddestination type field 0x05. The WM manager may generate a URLhttp://remoteserver.com using the URL protocol type field value and theURL value. In addition, a URL http://remoteserver.com?cid=123456&t=5005may be finally generated using the content ID and the timestamp value.The WM manager may make a request using the final URL (s53070).

The remote server may receive the request and transmit the URL of therelated application suitable for the broadcast program to the WM manager(s53080). The WM manager may send the received URL of the application tothe browser and launch the application (s53090).

Similarly to the above description, the WM detector and the WM managerof the terminal may be combined to perform the functions thereof in onemodule. In this case, steps s53050, s53060, s53070 and s53090 may beperformed in one module.

FIG. 26 is a diagram showing the structure of data to be inserted into aWM according to embodiment #3 of the present invention.

The present invention proposes a delivery type field as one of datawhich can be delivered via a watermarking scheme. In addition, thepresent invention proposes an efficient data structure of a deliverytype field.

In order to reduce deterioration in quality of audio/video content dueto increase in amount of data inserted into the WM, the WM may bedivided and inserted. In order to indicate whether the WM is divided andinserted, a delivery type field may be used. Via the delivery typefield, it may be determined whether one WM or several WMs are detectedin order to acquire broadcast related information.

If the delivery type field has a value of 0, this may mean that all datais inserted into one WM and transmitted. If the delivery type field hasa value of 1, this may mean that data is divided and inserted intoseveral WMs and transmitted.

In the present embodiment, the value of the delivery type field is 0. Inthis case, the data structure of the WM may be configured in the form ofattaching the delivery type field to the above-described data structure.Although the delivery type field is located at a foremost part in thepresent invention, the delivery type field may be located elsewhere.

The WM manager or the WM detector may parse the WM by referring to thelength of the WM if the delivery type field has a value of 0. At thistime, the length of the WM may be computed in consideration of thenumber of bits of a predetermined field. For example, as describedabove, the length of the event field may be 3 bits. The size of thecontent ID and the URL may be changed but the number of bits may berestricted according to embodiments.

FIG. 27 is a diagram showing the structure of data to be inserted into aWM according to embodiment #4 of the present invention.

In the present embodiment, the value of the delivery type field maybe 1. In this case, several fields may be added to the data structure ofthe WM.

A WMId field serves as an identifier for identifying a WM. If data isdivided into several WMs and transmitted, the WM detector needs toidentify each WM having divided data. At this time, the WMs each havingthe divided data may have the same WMId field value. The WMId field mayhave a size of 8 bits.

A block number field may indicate an identification number of a currentWM among the WMs each having divided data. The values of the WMs eachhaving divided data may increase by 1 according to order of transmissionthereof. For example, in the case of a first WM among the WMs eachhaving divided data, the value of the block number field may be 0x00. Asecond WM, a third WM and subsequent WMs thereof may have values of0x01, 0x02, . . . . The block number field may have a size of 8 bits.

A last block number field may indicate an identification number of alast WM among WMs each having divided data. The WM detector or the WMmanager may collect and parse the detected WMs until the value of theabove-described block number field becomes equal to that of the lastblock number field. The last block number field may have a size of 8bits.

A block length field may indicate a total length of the WM. Here, the WMmeans one of the WMs each having divided data. The block length fieldmay have a size of 7 bits.

A content ID flag field may indicate whether a content ID is included inpayload of a current WM among WMs each having divided data. If thecontent ID is included, the content ID flag field may be set to 1 and,otherwise, may be set to 0. The content ID flag field may have a size of1 bit.

An event flag field may indicate whether an event field is included inpayload of a current WM among WMs each having divided data. If the eventfield is included, the event flag field may be set to 1 and, otherwise,may be set to 0. The event flag field may have a size of 1 bit.

A destination flag field may indicate whether a destination type fieldis included in payload of a current WM among WMs each having divideddata. If the destination type field is included, the destination flagfield may be set to 1 and, otherwise, may be set to 0. The destinationflag field may have a size of 1 bit.

A URL protocol flag field may indicate whether a URL protocol type fieldis included in payload of a current WM among WMs each having divideddata. If the URL protocol type field is included, the URL protocol flagfield may be set to 1 and, otherwise, may be set to 0. The URL protocolflag field may have a size of 1 bit.

A URL flag field may indicate whether URL information is included inpayload of a current WM among WMs each having divided data. If the URLinformation is included, the URL flag field may be set to 1 and,otherwise, may be set to 0. The URL flag field may have a size of 1 bit.

The payload may include real data in addition to the above-describedfields.

If data is divided into several WMs and transmitted, it is necessary toknow information about when each WM is inserted. In this case, accordingto embodiments, a timestamp may be inserted into each WM. At this time,a timestamp type field may also be inserted into the WM, into which thetimestamp is inserted, in order to know when the WM is inserted.Alternatively, according to embodiments, the receiver may store and useWM timestamp type information. The receiver may perform timesynchronization based on a first timestamp, a last timestamp or eachtimestamp.

If data is divided into several WMs and transmitted, the size of each WMmay be adjusted using the flag fields. As described above, if the amountof data transmitted by the WM increases, the quality of audio/videocontent may be influenced. Accordingly, the size of the WM inserted intoa frame may be adjusted according to the transmitted audio/video frame.At this time, the size of the WM may be adjusted by the above-describedflag fields.

For example, assume that any one of video frames of content has a blackscreen only. If a scene is switched according to content, one videoframe having a black screen only may be inserted. In this video frame,the quality of content may not deteriorate even when a large amount ofWMs is inserted. That is, a user does not sense deterioration in contentquality. In this case, A WM having a large amount of data may beinserted into this video frame. At this time, most of the values of theflag fields of the WM inserted into the video frame may be 1. This isbecause the WM have most of the fields. In particular, a URL fieldhaving a large amount of data may be included in that WM. Therefore, arelatively small amount of data may be inserted into other video frames.The amount of data inserted into the WM may be changed according todesigner's intention.

FIG. 28 is a diagram showing the structure of data to be inserted into afirst WM according to embodiment #4 of the present invention.

In the present embodiment, if the value of the delivery type field is 1,that is, if data is divided into several WMs and transmitted, thestructure of a first WM may be equal to that shown in FIG. 28.

Among WMs each having divided data, a first WM may have a block numberfield value of 0x00. According to embodiments, if the value of the blocknumber field is differently used, the shown WM may not be a first WM.

The receiver may detect the first WM. The detected WM may be parsed bythe WM manager. At this time, it can be seen that the delivery typefield value of the WM is 1 and the value of the block number field isdifferent from that of the last block number field. Accordingly, the WMmanager may store the parsed information until the remaining WM having aWMID of 0x00 is received. In particular, atsc.org which is URLinformation may also be stored. Since the value of the last block numberfield is 0x01, when one WM is further received in the future, all WMshaving a WMID of 0x00 may be received.

In the present embodiment, all the values of the flag fields are 1.Accordingly, it can be seen that information such as the event field isincluded in the payload of this WM. In addition, since the timestampvalue is 5005, a time corresponding to a part, into which this WM isinserted, may be 5.005 seconds.

FIG. 29 is a diagram showing the structure of data to be inserted into asecond WM according to embodiment #4 of the present invention.

In the present embodiment, if the value of the delivery type field is 1,that is, if data is divided into several WMs and transmitted, thestructure of a second WM may be equal to that shown in FIG. 35.

Among WMs each having divided data, a second WM may have a block numberfield value of 0x01. According to embodiments, if the value of the blocknumber field is differently used, the shown WM may not be a second WM.

The receiver may detect the second WM. The WM manager may parse thedetected second WM. At this time, since the value of the block numberfield is equal to that of the last block number field, it can be seenthat this WM is a last WM of the WMs having a WMId value of 0x00.

Among the flag fields, since only the value of the URL flag is 1, it canbe seen that URL information is included. Since the value of the blocknumber field is 0x01, this information may be combined with alreadystored information. In particular, the already stored atsc.org part andthe /apps/app1.html part included in the second WM may be combined. Inaddition, in the already stored information, since the value of the URLprotocol type field is 001, the finally combined URL may behttp://atsc.org/apps/app1.html. This URL may be launched via thisbrowser.

According to the second WM, a time corresponding to a part, into whichthe second WM is inserted, may be 10.005 seconds. The receiver mayperform time synchronization based on 5.005 seconds of the first WM ormay perform time synchronization based on 10.005 seconds of the last WM.In the present embodiment, the WMs are transmitted twice at an intervalof 5 seconds. Since only audio/video may be transmitted during 5 secondsfor which the WM is not delivered, deterioration in quality of contentmay be prevented. That is, even when data is divided into several WMsand transmitted, quality deterioration may be reduced. A time when theWM is divided and inserted may be changed according to embodiments.

FIG. 30 is a flowchart illustrating a process of processing thestructure of data to be inserted into a WM according to embodiment #4 ofthe present invention.

Step s58010 of, at the service provider, delivering content to the WMinserter, step s58020 of, at the WM inserter, inserting the receivedcontent into the WM #1, step s58030 of, at the WM inserter, transmittingthe content, into which the WM #1 is inserted, step s58040 of, at theSTB, receiving the content, into which the WM #1 is inserted, andoutputting the incompressible A/V data, and step s58050 of, at the WMdetector, detecting the WM #1 may be equal to the above-described steps.

WM #1 means one of WMs into which divided data is inserted and may be afirst WM in embodiment #4 of the present invention. As described above,the block number field of this WM is 0x00 and URL information may beatsc.org.

The WM manager may parse and store detected WM #1 (s58060). At thistime, the WM manager may perform parsing by referring to the number ofbits of each field and the total length of the WM. Since the value ofthe block number field is different from the value of the last blocknumber field and the value of the delivery type field is 1, the WMmanager may parse and store the WM and then wait for a next WM.

Here, step s58070 of, at the service provider, delivering the content tothe WM inserter, step s58080 of, at the WM inserter, inserting thereceived content to WM #2, step s58090 of, at the WM inserter,transmitting the content, into which WM #2 is inserted, step s58100 of,at the STB, receiving the content, into which WM #2 is inserted, andoutputting incompressible A/V data and/or step s58110 of, at the WMdetector, detecting WM #2 may be equal to the above-described steps.

WM #2 means one of WMs into which divided data is inserted and may be asecond WM in embodiment #4 of the present invention. As described above,the block number field of this WM is 0x01 and URL information may be/apps/app1.html.

The WM manager may parse and store detected WM #2 (s58120). Theinformation obtained by parsing WM #2 and the information obtained byparsing already stored WM #1 may be combined to generate an entire URL(s58130). In this case, the entire URL may behttp://atsc.org/apps/app1.html as described above.

Step s58140 of, at the WM manager, delivering related data to thecompanion device protocol module of the receiver according to thedestination type field and step s58150 of, at the companion deviceprotocol module, delivering related data to the companion deviceaccording to the destination type field may be equal to theabove-described steps.

The destination type field may be delivered by WM #1 as described above.This is because the destination flag field value of the first WM ofembodiment #4 of the present invention is 1. As described above, thisdestination type field value may be parsed and stored. Since thedestination type field value is 0x02, this may indicate data for asmartphone.

The companion device protocol module may communicate with the companiondevice to process the related information, as described above. Asdescribed above, the WM detector and the WM manager may be combined. Thecombined module may perform the functions of the WM detector and the WMmanager.

FIG. 31 is a diagram showing the structure of a watermark based imagedisplay apparatus according to another embodiment of the presentinvention.

This embodiment is similar to the structure of the above-describedwatermark based image display apparatus, except that a WM manager t59010and a companion device protocol module t59020 are added under awatermark extractor s59030. The remaining modules may be equal to theabove-described modules.

The watermark extractor t59030 may correspond to the above-described WMdetector. The watermark extractor t59030 may be equal to the modulehaving the same name as that of the structure of the above-describedwatermark based image display apparatus. The WM manager t59010 maycorrespond to the above-described WM manager and the companion deviceprotocol module t59020 may correspond to the above-described companiondevice protocol module. Operations of the modules have been describedabove.

FIG. 32 is a diagram showing the structure of a watermark payloadaccording to another embodiment of the present invention.

The watermark payload of the shown embodiment may include domain typeinformation, server URL information, timestamp information and/ortrigger type information. In some embodiments, the shown watermarkpayload may be used as an audio or video watermark. Here, the watermarkmay be referred to as a WM. In some embodiments, the WM payload may havea size of 50 bits and a WM system may deliver 50 bits at an interval of1.5 seconds.

The domain type information may indicate the type of the WM payload. Thedomain type information may indicate how the sizes of the server URLinformation and timestamp information of the payload are assigned.According to the domain type information, there is a trade-off in ascope of uniqueness between a server code of a server URL field and aninterval code of a timestamp field. The domain type information mayindicate whether the payload has a small domain, a medium domain or alarge domain according to the size assigned to the field. In someembodiments, the domain type information may have a size of 1 bit. Inthis case, the domain type information may indicate whether the payloadhas a small domain or a large domain.

The server URL information may include a server code. This server codemay be a value for identifying a server operating as a starting pointfor supplementary content acquisition. Sever URL information or servercode may be in the format of an Internet address or IP address capableof acquiring supplementary content or a specific code mapped to such anaddress. It is possible to access a URL which may be confirmed throughthe server URL information to acquire a variety of supplementarycontent.

The supplementary content may mean content which may be provided to aviewer in addition to services/content currently transmitted from anMVPD to a receiver. The supplementary content may include services,content, timeline, application data, alternate components or applicationrelated information. The supplementary content may be referred to asinteractive service information. In addition, the supplementary contentmay include application property information for providing aninteractive service of a broadcast service/content. In addition, thesupplementary content may include event information of a specificapplication. Here, the event information may be notification orsignaling information for initiating actions to be performed by theapplication.

The timestamp information may include an interval code. This intervalcode may be a value for identifying the interval of the content intowhich the payload is embedded. The timestamp information or the intervalcode may identify the embedding interval of the payload or may identifytransmission time information of the content into which the WM packet orWM payload is embedded or how many WM packets or WM payloads areembedded. When it is identified how many WM packet or WM payloads areembedded, the time interval between WMs may be predetermined. In someembodiments, the timestamp information may be referred to as intervalinformation.

The trigger type information may signal when an event is available.Changing the value of the trigger type information within consecutive WMpayloads may indicate that an event is available/acquired from an eventserver. Here, the event may be the above-described event information.Here, the event may be a dynamic event. The dynamic event may mean anevent, the start time of which is known at the last minute. For example,event information of a live broadcast service may be a dynamic event.Here, the event server is a dynamic event server and may be an HTTPserver. In some embodiments, trigger type information may be referred toas query information, a query flag, etc.

That is, the trigger type information may indicate whether a URLaccording to server URL information needs to be accessed. In someembodiments, the trigger type information may indicate whetherapplication property information is acquired or whether eventinformation is acquired upon accessing the URL. The event information istime sensitive information and thus needs to be distinguished from theapplication property information, in order to prevent necessary timesensitive information from not being acquired by unnecessarily usingresources to acquire non-time-sensitive information. In someembodiments, the trigger type information may indicate whether theapplication property information to be acquired is changed, which willbe described in detail below.

In some embodiments, the server URL information and the timestampinformation may have a size of 30 bits or 17 bits (small domain type), asize of 22 bits or 25 bits (medium domain type) and a size of 18 bits or29 bits (large domain type). In some embodiments, these values may bechanged. In this case, the small domain may have one billion servercodes and an interval code of about 54.6 hours, the medium domain mayhave 4.2 million server codes and an interval code of about 1.59 yearsand the large domain may have 262,144 server codes and an interval codeof about 25.5 years.

In some embodiments, the server URL information and the timestampinformation may have a size of 31 bits or 17 bits (small domain type)and a size of 23 bits or 25 bits (large domain type). In this case, thedomain type information may have a size of 1 bit and the trigger typeinformation may have a size of 1 bit. In some embodiments, these valuesmay be changed.

In some embodiments, the trigger type information of the shown WMpayload may have a size of 2 bits. If the trigger type information is00, this indicates that the application property information may beacquired by accessing the server URL and this property information isnot changed as compared to the property information capable of beingacquired using the server URL of a previous WM. If the trigger typeinformation is 01, this indicates that the application propertyinformation may be acquired by accessing the server URL and thisproperty information is changed as compared to the property informationcapable of being acquired using the server URL of the previous WM. Ifthe trigger type information is 10, this indicates that eventinformation may be acquired by accessing the server URL. The triggertype information of 11 may be reserved for future use.

In some embodiments, if the trigger type information is assigned 2 bits,the meaning of the value of the trigger type information may be changed.For example, if the trigger type information is 00, this indicates thatthere is no additional app, component or information capable of beingacquired by making a request from a server at this interval. In thiscase, a request (query) may not be sent to the server. If the triggertype information is 01, this indicates that there is an additional app,component or information capable of being acquired by making a requestfrom a server at this interval. In this case, a request (query) may besent to the server. If the trigger type information is 10, thisindicates that event information may be acquired by accessing the serverURL. Accordingly, in this case, even when a request is made recently, arequest should be made again. The trigger type information of 11 may bereserved for future use.

In some embodiments, the structures of the above-described WM payloadsmay be combined. In addition, in some embodiments, the assigned sizes ofthe information of the above-described payloads may be combined. Forexample, trigger type information of 1 bit or trigger type informationof 2 bits may be combined, for the sizes of the server URL informationand the timestamp information according to the small, medium and largedomains. In addition, domain type information of 1 bit or the domaintype information of 2 bits may be combined, for each case.

FIG. 33 is a diagram showing change in watermark payload structure usingservice/content information according to one embodiment of the presentinvention.

Service information and/or content information may be added to anddelivered in each WM payload structure or each combinable WM payloadstructure. Here, the service information may be related to a service,into which the WM is embedded. This service information may be in theformat of a service ID or channel ID. When the service information isincluded and delivered in the WM payload, a server may selectivelyprovide only supplementary content (interactive service) of a specificservice/channel. In addition, when a service/channel which is beingviewed is changed, an interactive service of a previous service/channelmay be rapidly finished. Here, the content information may be related tocontent, into which the WM is embedded. The content information may bein the format of a content ID. When the content information is includedand delivered in the WM payload, the server may selectively providesupplementary content (interactive service) of specific content.

In the shown embodiment (t502010), service information and/or contentinformation is added to one of the above-described WM payloads. In theshown embodiment (t502020), the above-described WM payload structure isminimized and then service information and/or content information areadded. In this case, the domain type information is omitted and thesizes of the server URL information, the timestamp information and thetrigger type information are reduced to 18 bits, 17 bits and 2 bits,respectively. In the two embodiments, the service information andcontent information may have arbitrary sizes (x and y bits) according toa related broadcast system. In some embodiments, only one of the serviceinformation and the content information may be added.

FIG. 34 is a diagram showing change in watermark payload structure usingan NSC field according to one embodiment of the present invention.

Each of the above-described WM payload structures may be changed to adda no supplemental content (NSC) field. The NSC field may indicatewhether supplementary content is available. The NSC field may operate asa 1-bit flag. The supplementary content has been described above.

1 bit for the NSC field may be acquired by reducing the size of theabove-described domain type information. In some embodiments, the sizeof the domain type information may be reduced to 1 bit. As describedabove, the domain type information may include the type of the WMpayload. In this case, the domain type information may indicate whetherthe WM payload is a small domain or a large domain. That is, if twotypes of domain are sufficient, 1 bit of the domain type information isassigned to the NSC field to indicate whether supplementary content isavailable. In some embodiments, the NSC field may be added to theabove-described WM payload structure without reducing the size of thedomain type information.

In this embodiment, in the small domain, the server URL field may have asize of 22 bits and the timestamp field may have a size of 25 bits. Inthe large domain, the server URL field may have a size of 18 bits andthe timestamp field may have a size of 29 bits. The small domain mayhave about 4.2 million server codes and an interval code of about 1.59years and the large domain may have about 262,144 server codes and aninterval code of about 25.5 years. Here, the trigger type informationmay have a size of 1 bit or 2 bits.

According to the information on the WM payload, the receiver may send arequest (query) to the server. The request may be sent (1) when thereceiver receives (tunes) a first watermarked segment and makes arequest, (2) when a request is further made according to requestinformation of supplementary content and (3) when a request is madeaccording to the above-described trigger type information.

According to addition of the NSC field, a request may not be made whensupplementary content is not present. For example, when the receiverfirst receives a watermark, a request may not be made. Accordingly, inchannel surfing, addition of the NSC field may be efficient. Inaddition, even when supplementary content is not present, theservice/content may be marked (watermarked) in order to report serviceusage. Even in this case, addition of the NSC field may be efficient. Inparticular, addition of the NSC field may be further efficient in amechanism for delivering a report for storage & usage. That is,generally, addition of the NSC field may be efficient when a largeamount of content is marked (watermarked) but supplementary content isnot present. In addition, the continuously watermarked content may bepreferable in an SMPTE open ID. In this case, continuously markedcontent may aid two SDOs in determining a common WM solution.

FIG. 35 is a diagram showing a watermark payload structure for linkingvideo and audio watermarks according to one embodiment of the presentinvention.

The present invention proposes a method of simultaneously embedding avideo WM and an audio WM as one embodiment. To this end, a portion of avideo WM payload may be assigned to an audio WM payload. A portion(e.g., 50 bits) of the video WM payload (e.g., 30 to 60 bytes) may carryduplicate information of the audio WM payload. The duplicate informationmay be equal to the information of the audio WM payload and a copy ofthe audio WM payload.

In addition, the video WM and the audio WM may be synchronized andtransmitted. The video WM may include at least one or more messageblocks and one of the message blocks may have the WM payload of thestructure. In this case, the audio WM embedded in the audio of theservice/content may have the same WM payload as the video WM. At thistime, a first video frame of a message block for delivering a video WMpayload may be time aligned with a first part of an audio WMcorresponding thereto. In some embodiments, the video WM and the audioWM may be time aligned within a predetermined error. In someembodiments, the message block may include a plurality of video framesand each video frame may have the same video WM payload which isrepeated. In some embodiments, a portion of the audio WM payload may beassigned to carry the copy of the video WM payload.

For example, a problem may occur when a user retrieves and displays anelectronic service guide (ESG) from an MVPD set top box (STB). First,only when the audio WM is used, although the ESG is displayed, audio maybe continuously played. However, an audio WM client may not know thatthe ESG is being displayed. Accordingly, an application may becontinuously executed and graphics may overlap the ESG, thereby causingdisturbance.

In addition, if only the video WM is used, when the ESG is displayed,the video WM client may recognize that the WM disappears and judge thata viewer has changed channels or has completed an interactive event.Accordingly, although the viewer tries to turn the ESG off withoutchanging the channels and to resume the interactive service at apredetermined point, the application may be finished by the video WMclient.

Accordingly, the audio WM and the video WM are efficiently used intandem. Unlike the audio WM, the video WM may inform the receiver thatmain video is not focused upon on the screen (the ESG is used). Inaddition, unlike the video WM, the audio WM may continuously provide WMinformation while the ESG is used. Accordingly, the receiver may trackwhether the WMs or the related supplementary content are changed whileoperating the ESG.

Accordingly, the ESG being displayed on the screen may be recognized bythe video WM and appropriate receiver operation may be continuouslyperformed by the audio WM. For example, if the application does notprovide graphics (e.g., background app), the application may becontinuously executed regardless of the ESG. For example, if theapplication provides graphics, the application may be suppressed untilthe ESG disappears. For example, when the application receives an event,the receiver may process the event on the background until the ESGdisappears. That is, this problem may be solved by linking the audio WMand the video WM.

FIG. 36 is a diagram showing operation using linked video and audiowatermarks according to one embodiment of the present invention.

First, operation when an ESG is displayed on a screen by a user will bedescribed. First, an original service/content may be delivered from abroadcast station to an MVPD such as an STB (t505010). An external inputsource such as an STB or a cable may deliver the originalservice/content to a receiver (t505020). Here, the delivered AV contentis in an uncompressed state and may have the linked audio WM and videoWM. The receiver may sense the audio WM and the video WM and performoperation corresponding thereto.

Here, the user may request an ESG from a remote controller of the STB(t505030). The STB may display the ESG on a TV screen (t505040). The ESGmay be overlaid on the AV content which is being played back. The TVreceiver may sense the audio WM but may not sense the video WM(t505050). The receiver may recognize that main video content is coveredwith the other graphics such as ESG, access the linked audio WM andseamlessly perform necessary operation.

Next, operation when the user mutes content will be described. Stepst505010 to t505020 of the TV receiver receiving AV content through theSTB have been described above. Here, the user may request mute from theremote controller of the STB (t505030). The STB may mute the AV content(t505040). The TV receiver may sense the video WM but may not sense theaudio WM (t505050). The receiver may recognize that the main audiocontent is muted and acquire audio WM payload data through the linkedvideo WM payload. Therefore, the receiver can seamlessly performnecessary operation.

FIG. 37 is a diagram illustrating a video WM payload format according toan embodiment of the present invention.

Referring to (a), configuration fields of a video WM payload (or videowatermark) according to an embodiment of the present invention areshown.

The video WM payload may include a run_in_pattern field, at least onewm_message( ) field, at least one zero_pad field and/or an AudioWMduplicate field (or audio watermark).

The run_in_pattern field indicates that a video line includes a WM(watermark) having a specific format. For example, the run_in_patternfield is 16 bits and may have a value of “0xEB52”.

The wm_message( ) field may include a WM message and/or a WM messageblock. The WM message may include all data. The WM message may includeat least one WM message block. The WM message block may be a fragment ofthe WM message. For example, the wm_message( ) field can be referred toas the aforementioned message block. In addition, the WM message mayinclude a video WM and/or an audio WM.

The zero_pad field may have a value of “0” used to pad the end of aframe.

The AudioWM duplicate field may include an audio WM and/or an audio WMpayload. For example, the AudioWM duplicate field may includeinformation which is a duplicate of the audio WM payload. This duplicateinformation is the same as information of the audio WM payload and maybe a copy. The AudioWM duplicate field can be included in a video WM andmay be included in the wm_message( ) field of the video WM payload.

Hereinafter, the wm_message( ) field will be described in more detail.

For example, the wm_message( ) field may include a wm_message_id field,a wm_message_block_length field, a wm_message_version field, afragment_number field, a last_fragment field and/or a wm_message_bytes() field.

The wm_message_id field can identify the syntax and/or semantics of databytes transmitted in a message block. The wm_message_id field may bereferred to as a WMId field. For example, the wm_message_id field canindicate one of a content ID message, a presentation time message, a URImessage, a vpl_message, an audio WM, an event message and an emergencyalert message.

The wm_message_block_length field can specify the number of bytesremaining in the wm_message( ) field. The wm_message_block_length fieldmay be referred to as a block length field.

The fragment_number field can indicate the number of a current messagefragment or a value obtained by subtracting “1” from the number of thecurrent message fragment. The fragment_number field may be referred toas a block number field.

The last_fragment field can indicate the fragment number of the lastfragment used to transmit a complete WM message (or wm_message( )). Thelast_fragment field may be referred to as a last block number field.

The wm_message_bytes( ) field may be referred to as a payload includingactual data. When the last_fragment field is “0”, the wm_message_bytes() field can include a WM message of a complete instance identified bythe wm_message_id field. When the last_fragment field is not “0”, thewm_message_bytes( ) field can include fragment data of the correspondingWM message. For example, the wm_message_bytes( ) field can include atleast one of a content ID message, a presentation time message, a URImessage, a vpl_message (or audio WM), an event message and an emergencyalert message.

The wm_message version field can indicate the version of a WM message.

Referring to (b), a video WM payload according to an embodiment of thepresent invention is shown.

One video WM payload may include all pieces of data.

For example, the video WM payload can include a run_in_pattern field, atleast one wm_message( ) field (wm_message_0, . . . , wm_message_N−1)and/or an AudioWM duplicate field. The wm_message( ) field may include acomplete WM message or at least one divided WM message block. In thiscase, although one video WM payload can include all pieces of data, thesize of the video WM payload may increase.

In addition, one video WM payload may be divided into a plurality ofvideo WM payloads. For example, a divided video WM payload may include arun_in_pattern field, at least one wm_message( ) field (wm_message_0, .. . , wm_message_N−1) and/or an AudioWM duplicate field. The wm_message() field may include at least one divided WM message block. The AudioWMduplicate field may be included in a divided video WM payload or atleast one divided WM message block.

In this case, each divided video WM payload can include divided data ofa WM message and thus the size of each video WM payload can bedecreased.

FIG. 38 illustrates a receiver operation when a user displays an ESG ona screen while a visible application is executed according to anembodiment of the present invention.

First, an original service/content may be received through an MVPD suchas an STB from a broadcaster. An external input source such as an STBand a cable can deliver the original service/content to a receiver.Here, delivered AV content may be in an uncompressed state and have alinked audio WM and video WM. The receiver can detect the audio WM andthe video WM and perform operations according thereto.

Then, a visible application indicated by the video WM can be operated inthe receiver (CS400100). The visible application may be an applicationwhich visually appears on a screen of the receiver.

Thereafter, the user may request an ESG using a remote controller of theSTB. For example, the user can push a button of the remote controller ofthe STB in order to view the ESG.

Then, the ESG is displayed on the screen (e.g., TV screen) of thereceiver (CS400200). The ESG may be overlaid on the AV content beingreproduced. In this case, the receiver detects the audio WM but may notdetect the video WM.

Subsequently, the receiver may determine whether both the audio WM andthe video WM are scheduled to be transmitted (CS400300).

When both the audio WM and the video WM are not scheduled to betransmitted, the receiver needs to stop the visible application. Afterthe ESG disappears, the receiver can start the operation of thecorresponding application upon detection of the video WM again(CS400400).

When both the audio WM and the video WM are scheduled to be transmitted,the receiver can continuously receive the audio WM (CS400500). Inaddition, the receiver can be aware that the video WM has beentemporarily covered and channel change does not occur. That is, thereceiver can recognize that channel change does not occur on the basisof the continuously transmitted audio WM.

Then, the receiver can temporarily pause or suspend the operation of thevisible application in order to resume the operation of the visibleapplication after the ESG disappears (CS400600). Here, a state of theapplication at the moment when the operation of the application ispaused or suspended may be stored and used when the operation of theapplication is resumed.

Accordingly, the user may not operate an application (e.g., visibleapplication) indicated by the audio WM on the ESG or may not display theapplication on the screen upon recognizing that main video content hasbeen covered by other graphics such as the ESG.

FIG. 39 illustrates a receiver operation when a visible application isexecuted after an ESG is displayed on a screen according to anembodiment of the present invention.

A basic operation of the receiver may include the aforementioneddescription. The following description is based on a difference from theabove-described receiver operation.

Distinguished from the above description, a visible application is notexecuted in the receiver (CS410100). A user may request an ESG using aremote controller of an STB.

Then, the ESG is displayed on the screen (e.g., TV screen) of thereceiver (CS410200). The ESG may be overlaid on AV content beingreproduced. In this case, the receiver detects an audio WM but may notdetect a video WM.

Thereafter, the receiver requests execution of the visible applicationusing the audio WM (CS410300).

Subsequently, the receiver may determine whether both the audio WM andthe video WM are scheduled to be transmitted (CS410400).

When both the audio WM and the video WM are not scheduled to betransmitted, the receiver needs to execute the requested visibleapplication on the ESG (CS410500). That is, the visible application canbe started after the ESG is displayed on the screen.

When both the audio WM and the video WM are scheduled to be transmitted,the receiver can continuously receive the audio WM (CS410600). Inaddition, the receiver can be aware that the video WM has beentemporarily covered and channel change does not occur. That is, thereceiver can recognize that channel change does not occur on the basisof the continuously transmitted audio WM.

Then, the receiver may not start the visible application in order tostart the visible application after the ESG disappears (CS410700).

Accordingly, the receiver may not operate an application (e.g., visibleapplication) indicated by the audio WM on the ESG or may not display theapplication on the screen upon recognizing that main video content hasbeen covered by other graphics such as the ESG.

FIG. 40 illustrates a receiver operation when a receiver is muted by auser while an audible application is executed according to an embodimentof the present invention.

A basic operation of the receiver may include the aforementioneddescription. The following description is based on a difference from theabove-described receiver operation.

An audible application indicated by an audio WM can be executed in thereceiver (CS420100). The audible application may be an application whichgenerates sound through a speaker of the receiver.

The user may request muting using a remote controller of an STB. Forexample, the user can push a button of the remote controller of the STBfor muting.

Then, the receiver (e.g., TV) is muted (CS420200). The receiver detectsa video WM but may not detect an audio WM.

Subsequently, the receiver may determine whether both the audio WM andthe video WM are scheduled to be transmitted (CS420300).

When both the audio WM and the video WM are not scheduled to betransmitted, the receiver needs to stop the audible application beingexecuted. Then, the receiver can start the corresponding applicationupon detection of the audio WM again after muting is turned off(CS420400).

When both the audio WM and the video WM are scheduled to be transmitted,the receiver can continuously receive the video WM (CS420500). Inaddition, the receiver can be aware that the audio WM has temporarilydisappeared due to muting and channel change does not occur. That is,the receiver can recognize that channel change does not occur on thebasis the continuously transmitted video WM.

To resume the operation after muting is off, the receiver maytemporarily pause or suspend the audible application being executed.Here, a state of the application at the moment when the operation of theapplication is paused or suspended may be stored and used when theoperation of the application is resumed.

Accordingly, the user may not generate sound of an application (e.g.,audible application) indicated by the video WM upon recognizing thatmain audio content is muted.

FIG. 41 illustrates a receiver operation when an audible application isexecuted after muting according to an embodiment of the presentinvention.

A basic operation of the receiver may include the aforementioneddescription. The following description is based on a difference from theabove-described receiver operation.

Distinguished from the above description, an audible application is notexecuted in the receiver (CS430100). A user may request muting using aremote controller of an STB.

Then, the TV is muted (CS430200). In this case, the receiver detects avideo WM but may not detect an audio WM.

The receiver requests execution of the audible application using thevideo WM (CS430300).

Then, the receiver may determine whether both the audio WM and the videoWM are scheduled to be transmitted (CS430400).

When both the audio WM and the video WM are not scheduled to betransmitted, the receiver needs to execute the audible applicationalthough the user has muted the receiver and/or the STB (CS430500). Thatis, the audible application can be started after the receiver is muted.

When both the audio WM and the video WM are scheduled to be transmitted,the receiver can continuously receive the video WM (CS430600). Inaddition, the receiver can be aware that the audio WM has temporarilydisappeared due to muting and channel change does not occur. That is,the receiver can recognize that channel change does not occur on thebasis of the continuously transmitted video WM.

Subsequently, the receiver may not immediately start the audibleapplication in order to start the operation after muting is turned off.

Accordingly, the receiver may not generate sound of the application(e.g., audible application) indicated by the video WM upon recognizingthat main audio content has been muted.

FIG. 42 is a diagram describing advantages of a video WM including anaudio WM according to an embodiment of the present invention.

An application is executed in a receiver (CS440100). The receiver canreceive information related to the application such as attributes andevents through an audio WM. A user can push a button of the remotecontroller of the STB for muting.

Then, the receiver is muted (CS440200). For example, the receiver candetect the audio WM until a time index T1 but cannot detect the audio WMafter that point.

The receiver starts to receive the video WM from the next time index(i.e., T2) (CS440300). For example, the video WM can include the audioWM.

Accordingly, the receiver can provide a seamless application service onthe basis of the audio WM.

FIG. 43 is a diagram describing advantages of a video WM including anaudio WM according to another embodiment of the present invention.

An application is executed in a receiver (CS450100). The receiver canreceive information related to the application such as attributes andevents through a video WM. A user can push a button of the remotecontroller of the STB in order to display an ESG.

Then, the screen of the receiver is covered by the ESG (CS450200). Forexample, the receiver can detect the video WM until a time index T1 butcannot detect the video WM after that point.

The receiver starts to receive the audio WM from the next time index(i.e., T2) (CS450300). For example, the video WM can include the audioWM. Since the audio WM is transmitted through the video WM, the receivercan be aware of the audio WM segment immediately following the finallyreceived video WM segment. That is, the receiver can recognize the audioWM segment immediately following the finally received video WM segmenton the basis of the audio WM included in the finally received video WMand acquire the next audio WM which is separately received.

Accordingly, the receiver can provide a seamless application service onthe basis of the audio WM.

FIG. 44 is a diagram illustrating a wm_message( ) format according toanother embodiment of the present invention.

As described above, the wm_message( ) field may include an entire WMmessage or one WM message block. A WM message may include at least oneWM message block. A WM message block may be a WM message fragment. Whena specific WM message format is defined as not being fragmented or a WMmessage is delivered without being fragmented according to anembodiment, the wm_message( ) field can include the entire WM message. AWM message block may be called a WM message fragment.

There are various types of WM messages, for example, acontent_id_message( ) a channel_id_message( ), a uri_message( ), avpl_message( ), a dynamic_event_message( ) and/or an EA_message( ). WMmessages types may further include a presentation_time_message( ), adisplay_override_message( ), and the like.

The content_id_message( ) may be a WM message which delivers a contentID related to a corresponding program (=content). This WM message maydeliver information such as a major/minor channel number related to theservice of the corresponding content according to an embodiment.

The channel_id_message( ) may be a WM message which delivers a serviceID related to a corresponding channel (=service). Information includedin this WM message may be delivered through the content_id_message( )and this WM message may not be defined according to an embodiment.

The uri_message( ) may be a WM message used to deliver various types ofURIs. The uri_message( ) may include URI type information, a URI stringlength and/or a URI string. URIs of a signaling server for SLS delivery,an ESG server for ESG delivery, a usage reporting server, and a dynamicevent HTTP server/Web Socket server for dynamic event delivery can betransmitted.

The vpl_message( ) may be a WM message used for the aforementioned linkbetween a video WM and an audio WM. This WM message may be used todeliver the same WM payload as the aforementioned audio WM payload. Whenthis message is delivered along with video data, an audio WM can deliverthe same WM payload. A video WM payload and an audio WM payload havingsuch information may be called a vpl payload. These may be time-alignedas described. The aforementioned AudioWM duplicate can be omitted whenthe vpl_message( ) is used.

The dynamic_event_message( ) may be a WM message for delivering adynamic event and event signaling. This WM message may be called anevent message. Details will be described below.

The EA_message( ) may be a WM message which delivers emergency alert(EA) related information. This WM message may be called an EA message.

The presentation_time_message( ) may indicate a presentation time of avideo frame carrying the corresponding WM to the receiver. This WMmessage may include presentation time information and the like.

The display_override_message( ) may be a WM message which indicates aspecific interval in which audio needs to be reproduced without beingdeformed or a specific interval in which video needs to be reproducedwithout being obstructed by overlay graphics. This is for the purpose ofpreventing a case in which EA information is not effectively deliveredto a user when an EA is burned in audio/video.

The wm_message( ) according to the illustrated embodiment may include awm_message_id field, a wm_message_length field, a wm_message_sequencefield, a fragment_number field and/or a last_fragment field. The fieldsmay be the same as the aforementioned wm_message_id field, thewm_message_block_length field, the fragment_number field and thelast_fragment field.

The wm_message( ) according to the illustrated embodiment may includeone of WM messages such as the content_id_message( ), thechannel_id_message( ), the uri_message( ), the vpl_mesage( ), thedynamic_event_message( ), the presentation_time_message( ), thedisplay_override_message( ) and the EA_message( ) or one of WM messageblocks fragmented from the corresponding WM message according to thevalue of the wm_message_id. A WM message may be additionally added orsome of the aforementioned WM message may not be used according to anembodiment.

When a WM message is fragmented into two or more WM message blocks anddelivered, a wm_message( ) which delivers the last fragment may furtherinclude 32-bit CRC information that covers the entire WM message.

Although the aforementioned wm_message( ) has been described asinformation included in a video WM payload, it may be used asinformation included in an audio WM payload according to an embodiment.

FIG. 45 is a diagram illustrating a structure of an audio WM payloadaccording to another embodiment of the present invention.

An MVPD can signal an emergency alert in various ways. First, the MVPDcan forcibly tune an STB to an MVPD channel which displays an EAmessage. Otherwise, the MVPD can cause the current MVPD channel todisplay an EA message. This can be achieved when the MVPD and abroadcaster agree on retransmission. In this case, the MVPD may generatean EA message thereof and provide the same on content.

When the MVPD displays an EA message through the current broadcastchannel (i.e., when the MVPD does not forcibly tune the STB), a problemmay be generated if the user displays a personalized graphic overlay oradditional interactive content on the screen. To deliver an EA messageto a viewer without obstructing the viewer, the receiver may need torapidly take down overlays when EA is generated. Such take-down may needto be performed using a WM. Here, the overlays are graphic overlayswhich cover TV screens and may include application screens.

The aforementioned operation is based on the assumption that the EAmessage has been burned-in at a certain point upstream in an MVPDscenario. That is, the EA message has been embedded in video/audio mediadata as video or audio data. The aforementioned operation can beperformed by a transmission side or the MVPD.

In this case, the burned-in EA message may be obstructed by alternativeor supplementary content. For example, a WM may cause the receiver torender certain interactivity (play along in a game show, and the like).Such interactive graphics and audio may obstruct all or some EAmessages. In addition, a WM may cause a target advertisement to beembedded. In this case, audio and video from an HDMI may not be renderedto viewers. Accordingly, the burned-in EA message may not be provided toviewers.

In such cases, the receiver can continuously monitor content deliveredfrom an STB and the like in order to detect a WM change even duringplayback of a target advertisement. Accordingly, when a viewer changeschannels or when other situations occur and thus the receiver needs tostop the target advertisement or interactivity rendering, the receivercan be aware of such situation.

A video WM may include message type information about an EA message inorder to allow the receiver to rapidly recognize generation of EA and totake down a graphic overlay, target content or the like. An audio WMincludes a trigger flag such that the receiver can send a new query to aserver.

For example, how fast the receiver responds to a trigger can be detectedthrough the audio WM. With respect to the audio WM, the receiver mayneed to recognize a trigger in the WM first, request the trigger from aserver, parse a recovery file and then take down supplementary content.A total time required to perform such procedure may be changed accordingto network traffic, a connection speed, a random float time forspreading a query, and the like. However, a long period time is taken inan emergency situation when periods of time taken in respectiveprocesses are added.

To solve this problem, a 1-bit flag (EA flag) for EA may be added to theaudio WM. Upon detection of the EA flag, the receiver can stop allgraphic overlays and immediately return to a main broadcast program.Accordingly, a server query procedure and the like may be omitted. Inaddition, it is possible to reduce the dependence of a response time onan Internet connection situation. Of course, an additional query may beperformed as necessary.

The aforementioned solution may be achieved by methods other than themethod of adding the 1-bit EA flag. First, a WM insertion method isdivided into two types of processing, and when a WM is inserted, areceiver may be controlled to recognize an EA situation using specificprocessing. Alternatively, a duration in which an overlay and the likeneed to be in a state in which they have been taken down in order todisplay an EA message may be signaled instead of the 1-bit EA flag. Thismay be signaled through the aforementioned display_override_message( ).

Furthermore, other WM solutions may be provided. For example, an audioWM may be completely removed during EA. An WM extractor may interpretcomplete removal of the audio WM as channel change and thus all graphicscan be taken down. In this case, however, a viewer cannot execute anaudience measurement function. Alternatively, it is possible to preventsituations in which EA is missed according to obstruction by using theaudio WM along with the video WM all the time. In this case, however,the receiver must have both video/audio WM detectors.

The illustrated WM payload structure is an audio WM payload structurefor including the aforementioned EA flag. An illustrated embodimentt45010 is the aforementioned general audio WM payload structure havingno EA flag. A domain_type field has a size of 1 bit and indicateswhether the payload has a small domain or a large domain. Accordingly,the domain_type field can be followed by a server code and an intervalcode assigned bits. Ranges of the server code and the interval codeaccording to domain types are as shown (t45060). The server code and theinterval code may be followed by a 1-bit trigger flag (query flag). Thequery_flag can indicate whether there is dynamic event update, asdescribed above, and a server address for obtaining a dynamic event canbe delivered through a recovery file or a WM.

An illustrated embodiment t45020 may be a structure in which the payloadfurther includes an EA flag. The EA flag has been described above. TheEA flag can indicate that a burn-in EA message is currently displayedwhen set to 1. The EA flag can indicate that the burn-in EA message isnot displayed when set to 0. When the EA flag is set to 1, the receivercan immediately stop graphic overlay and output of additional audiocontent.

The EA flag may be positioned after the domain_type field according toan embodiment. An illustrated embodiment t45030 may be a structure inwhich the EA flag replaces the domain_type field. In this case, thedomain_type field does not exist and only one bit allocationconfiguration for the server code and the interval code can be used.

Illustrated embodiments t45040 and t45050 are tables showing payloadstructures including the aforementioned EA flag. They respectivelyrepresent payload structures in a small domain and a large domain. Theseembodiments are similar to the above-described embodiment t45020 butdiffer therefrom in that the EA flag is positioned after the query_flag.

FIG. 46 is a diagram illustrating a structure of an EA_message( )according to an embodiment of the present invention.

The EA_message( ) which provides information related to EA may bedelivered through a video WM. The video WM can provide a larger payloadthan an audio WM and thus is better for related information delivery. Inaddition, when the receiver detects that the wm_message_id value of thevideo WM is the EA_message( ) type, the receiver can stop (take down)all other overlays and return to main content. That is, thewm_message_id value may serve as the aforementioned EA flag. This may beperformed through an A flag, a WM processing scheme or thedisplay_override_message( ).

The EA_message( ) is one of the aforementioned WM messages. TheEA_message( ) may have various structures according to embodiments. TheEA_message( ) may be called an EA message according to context.

An EA_message( ) according to an illustrated embodiment t46010 mayinclude a CAP_message_ID_length field, a CAP_message_ID field, aCAP_message_url_length field, a CAP_message_url field, an expires field,a urgency field and/or a severity_certainty field.

The CAP_message_ID_length field can indicate the length of theCAP_message_ID field.

The CAP_message_ID field can indicate the ID of a CAP (Common AlertingProtocol) message provided through the CAP_message_url field. This fieldmay have the same value as the ID field in the CAP message.

The CAP_message_url_length field can indicate the length of theCAP_message_url field.

The CAP_message_url field can indicate a URL through which thecorresponding CAP message can be acquired.

The expires field can indicate the latest expiration date amongexpiration dates of info elements of the corresponding CAP message. Thiscan be represented on the basis of TAI (International Atomic Time).

The urgency field can indicate that urgency of a most urgent infoelement of the corresponding message is “immediate” when set to 1 andcan be set to 0 in other cases.

The severity_certainty field may be a code value derived from certaintyand severity information of the corresponding CAP message. The certaintyinformation can indicate which one of “unknown”/“unlikely”, “possible”,“likely” and “observed” corresponds to certainty of a correspondingemergency. The severity information can indicate which one of“unknown”/“minor”, “moderate”, “severe” and “extreme” corresponds toseverity of corresponding emergency.

An EA_message( ) according to an illustrated embodiment t46020 mayinclude an EA_present_flag field, an EA_present_version field, anEA_message_ID field, an EA_message_version field, anEA_message_text_length field and/or an EA_message_text field. TheEA_message( ) may further include an EA_rich_media_length field and/oran EA_rich_media_url field according to an embodiment.

The EA_present_flag field can indicate whether a wake_up bit on physicallayer signaling has been activated. This field can indicate that thewake_up bit has been activated, that is, EA exists when set to 1 andindicate that the wake-up bit is deactivated, that is, EA does not existwhen set to 0.

The EA_present_version field can indicate a version number of thewake-up bit. This may have a format defined in physical layer signaling.The EA_message_ID field can indicate the ID of a corresponding EAmessage. This may be the same as the aforementioned CAP_message_ID. TheEA_message_version field can indicate the version of the correspondingEA message. The EA_message_text_length field can indicate the length ofthe EA_message_text field. The EA_message_text field can provide abanner text value of the corresponding EA message. TheEA_rich_media_length field can indicate the length of theEA_rich_media_url field.

The EA_rich_media_url field can provide a URL through which rich mediarelated to the corresponding EA message. Rich media may refer to mediacontent including video and audio which represent emergency informationof the corresponding EA message.

The EA_message( ) according to an illustrated embodiment t46030 mayinclude an EA_present_flag field, an EA_present_version field, aCAP_message_ID field, a CAP_message_version field, aCAP_message_url_length field, a CAP_message_url field and/or anEA_message_overlay_area field.

The EA_present_flag field and the EA_present_version field have beendescribed above. The CAP_message_ID field, the CAP_message_versionfield, the CAP_message_url_length field and the CAP_message_url fieldmay be the same as the aforementioned EA_message_ID field,EA_message_version field, EA_message_url_length field and EA_message_urlfield.

The EA_message_overlay_area field can indicate an overlay area occupiedby a burn-in EA message on a receiver screen. For example, this fieldcan provide two positions of an upper left position and a lower rightposition to indicate a rectangular area covered by a burn-in banner.This field may also be included in EA messages in different forms fromthe embodiment t46030 as necessary.

An EA_message( ) according to an illustrated embodiment t46040 mayinclude an EA_present_flag field, an EA_present_version field, anEA_message_ID field, an EA_message_version field, anEA_message_text_length field, an EA_message_text field, anEA_rich_media_url_length field, an EA_rich_media_url field, anEAT_server_url_length field, an EAT_server_url field and/or anEAM_position field.

The EA_present_flag field, the EA_present_version field, theEA_message_ID field, the EA_message_version field, theEA_message_text_length field, the EA_message_text field, theEA_rich_media_url_length field and the EA_rich_media_url field have beendescribed above.

The EAT_server_url_length field can indicate the length of theEAT_server_url field.

The EAT_server_url field can indicate a URL for acquiring an EAT(Emergency Alert Table). The EAT may be a signaling table which providesinformation related to one or more EAs.

The EAM_position field can be the same as the aforementionedEA_message_overlay_area field.

FIG. 47 is a diagram illustrating an audio WM payload structureaccording to another embodiment of the present invention.

A DE (Dynamic Event) flag may be added to the aforementioned WM payloadstructure. This WM payload structure may be an audio WM payloadstructure. The DE flag can indicate presence of absence of a dynamicevent.

A broadcaster can indicate that there is a new event that needs to beacquired by the receiver through an MVPD in real time. Upon acquisitionof the DE flag, the receiver can recognize presence of the new dynamicevent. In addition, the receiver can access an event server to recognizewhether an event message for the corresponding event can be acquired.The receiver may already know the address of the event server through arecovery file or a WM.

An illustrated embodiment t47010 may be a structure in which the payloadfurther includes the DE flag. The DE flag has been described above. Whenthe DE flag is set to 1, the receiver can request an event message forthe corresponding event from the event server. Accordingly, the receivercan acquire the event message. That is, the receiver can immediatelystop all graphic overlays and additional audio content.

The DE flag may be positioned after the domain_type field according toan embodiment. The DE flag may serve as the aforementioned query_flagaccording to an embodiment. In this case, the DE flag is not used andthus is omitted in the WM payload structure and only the query_flag canbe used. Here, only the domain_type field, the server code field, theinterval code field and the query_flag field can be included in the WMpayload. When the query_flag indicates whether supplementary content(recovery file) has been updated, the query_flag and the DE flag maysimultaneously exist according to an embodiment.

Illustrated embodiments t47020 and t47030 are tables showing payloadstructures including the DE flag. They respectively show payloadstructures in a small domain and a large domain. These embodiments aresimilar to the aforementioned embodiment t47010 but differ therefrom inthat the DE flag is positioned after the query_flag.

FIG. 48 is a diagram illustrating a structure of adynamic_event_message( ) according to an embodiment of the presentinvention.

A dynamic event or an event may be delivered to the receiver throughvarious methods. First, whether a dynamic event is available throughquery_flag information of an audio WM is checked and then the receiversends a query to a server and receives an event message. The eventmessage may be a message which provides event signaling with respect tothe corresponding dynamic event. The event message may be the same asthe dynamic_event_message( ) which will be described below or may have asimilar format thereto.

Further, the dynamic_event_message( ) may be delivered to the receiveras one of WM messages of a video WM payload. As described above, thedynamic_event_message( ) is one of WM messages and can be fragmentedinto WM message blocks and delivered to the receiver.

The dynamic_event_message( ) is one of the aforementioned WM messages.The dynamic_event_message( ) may have various structures according toembodiments. The dynamic_event_message( ) may be called an event messageaccording to context.

A dynamic_event_message( ) according to an illustrated embodiment t48010may include a delivery_protocol_type field, a scheme_id_uri_strlenfield, a scheme_id_uri_string field, a value_strlen field, avalue_string field, a timescale field, a presentation_time field, apresentation_time_ms field, a duration field, an id field, a data_lengthfield, a data field, a reserved1_field_length field and/or a reserved1field.

The delivery_protocol_type field can indicate a delivery protocol of abroadcast service to which the corresponding dynamic event is applied.This field can indicate a ROUTE/DAH protocol or an MMT protocol.

The scheme_id_uri_strlen field can indicate the length of thescheme_id_uri_string field.

The scheme_id_uri_string field can indicate schemeIdUri information ofan event stream of the corresponding event. This information may be usedto identify an application related to the corresponding event accordingto an embodiment.

The value_strlen field can indicate the length of the value_stringfield.

The value_string field can indicate value information of an event streamof the corresponding event. This field can indicate actual values ofparameters defined according to the aforementioned schemeIdUriinformation.

The timescale field can indicate a timescale for an event stream of thecorresponding event.

The presentation_time field can indicate a presentation time of thecorresponding event. This time can be represented on the basis of TAI.

The presentation_time_ms field may be an offset in milliseconds based onthe presentation_time. This field can have values in the range of 0 to999. Accordingly, a correct actual presentation time of an event can beindicated in milliseconds through calculation ofpresentation_time+(presentation_time_ms/1000).

The duration field can indicate the duration of the corresponding event.This may be represented in the aforementioned timescale unit.

The id field can indicate the ID of the corresponding event.

The data_length field can indicate the length of the data field.

The data field can indicate data necessary to respond to thecorresponding event. This field may be omitted when such data is notnecessary.

The reserved1_field_length field can indicate the length of thereserved1 field. The reserved1 field may be a space reserved for futureuse.

A dynamic_event_message( ) according to an illustrated embodiment t48020may be modified from the dynamic_event_message( ) according to theaforementioned embodiment t48010.

In this embodiment, when service delivery is performed according to theROUTE/DASH protocol depending on a value of the delivery_protocol_type,the dynamic_event_message( ) may further include an MPD_id_length field,an MPD_id field, a period_id_length field and/or a period_id field. Whenservice delivery is performed according to the MMT protocol, thedynamic_event_message( ) may further include an asset_id_length field,an asset_id field and/or an MPU_seq_num field. Further, the meaning ofthe presentation_time field may be changed and the presentation_time_msfield may be omitted according to an embodiment.

The presentation_time field can indicate a presentation time of thecorresponding event. This can be represented according to timescale. Thepresentation_time field differs from the aforementionedpresentation_time field in that the former indicates a presentation timeas a value relative to a presentation time of a specific data unit onthe basis of the specific data unit instead of indicating thepresentation time on the basis of TAI.

When a service is delivered according to the ROUTE/DASH protocol, apresentation time can be indicated on the basis of a start time of aDASH period. An MPD may include a plurality of period elements. The DASHperiod which is a basis can be referenced by an MPD ID and a period IDwhich will be described below.

When a service is delivered according to the MMT protocol, apresentation time can be indicated on the basis of a start time of anMMT MPU (Media Processing Unit). An asset of the MMT can sequentiallyinclude a plurality of MPUs. An MPU which is a basis can be referencedby an asset ID and an MPU sequence number value which will be describedbelow.

The MPD_id_length field can indicate the length of the MPD_id field inbytes. The MPD_id field can identify an MPD including the aforementionedperiod which is a basis. Here, the period which is a basis refers to aperiod which provides a reference timeline of the corresponding event.

The period_id_length field can indicate the length of the period_idfield in bytes. The period_id_length field can identify theaforementioned period which is a basis within the MPD identified by theMPD_id field.

The asset_id_length field can indicate the length of the asset_id fieldin bytes. The asset_id field can identify an asset including theaforementioned MPU which is a basis. Here, the MPU which is a basisrefers to an MPU which provides a reference timeline of thecorresponding event.

The MPU_seq_num field can identify the aforementioned MPU which is abasis within the asset identified by the asset_id field.

FIG. 49 is a diagram illustrating a format of a recovery file accordingto an embodiment of the present invention.

Watermark segments refer to continuously marked intervals of content.Here, “marked” means “watermark-inserted”. Such intervals may include WMpayloads in neighboring cells. WM payloads embedded in neighboring cellsmay have the same server code value and also have sequentiallyincreasing interval code values.

A query_flag value may be changed between neighboring WM payloads withinone WM segment. This change can indicate whether a dynamic event isavailable in a server, as described above. Change of the query_flagvalue in neighboring WM payloads of one WM segment can indicate that adynamic event is available in a dynamic event server.

A recovery process will be described.

Redistribution setting refers to a situation in which a device receivesTV audio/video content from a certain input (input source). That is,redistribution setting may be a situation in which the device does notdirectly access broadcast streams from which the TV audio/video contentis derived. Here, the input may refer to an HDMI cable or a set-top box,for example. For example, a situation in which the device receives TVcontent from a cable, a satellite, an IPTV set-top box or the like maybe a redistribution situation.

The recovery process refers to a process through which a device inredistribution setting accesses supplementary content through broadband.Here, supplementary content can be provided by a broadcaster ofbroadcast content received through a cable or the like.

One of recovery processes which can be performed by a device inredistribution setting may be performed when the device receives anaudio WM from received content or receives a vpl message within a videoWM. Another recovery process may be performed when the device receives afull set of video WMs from received content.

A method of acquiring signaling information using an audio WM will bedescribed.

A device (receiver) may perform an HTTP request to a well-known server.This request can be performed using a server code and an interval codeof an audio WM as a part of a URL for the request.

The device can receive a recovery file as a response to the request. Therecovery file may include original presentation time information ofcontent at the beginning of the audio WM, information about a currentlyviewed service, and URL information which can be used to acquire a setof signaling files. These signaling files may be signaling filesnecessary to present and access supplementary content. These signalingfiles can be current from the date and time of the audio WM.

The device can acquire the set of the signaling files. The files mayinclude an MPD, an AIT, an EMT, an AEI, and the like.

The device can access supplementary content using the signaling filesand present the same. In this process, the time information of therecovery file can be used for synchronization between the supplementarycontent and audio/video received from a cable.

Each signaling file may have a “valid from” attribute, a “valid until”attribute and/or a “next URL” attribute related thereto. The “validfrom” attribute and/or the “valid until” attribute define validintervals of the corresponding signaling file. The “next URL” attributeindicates the URL of the corresponding signaling file of the schedulednext version. The device can acquire scheduled update of thecorresponding signaling file as necessary through the attributes.

The query flag of the audio WM can be used to signal availability of anevent. When such signaling is detected, the device can request thecorresponding event using the URL of the dynamic event server optionallyprovided through the recovery file.

Alternatively, the device can be connected to a Web Socket server toreceive a dynamic event. In this case, connection can be performed usinga WebSocket server URI which can be optionally provided through therecovery file.

Such an event may be intended for an application being executed in arun-time environment. Further, the event may signal availability ofunscheduled update for a signaling file. In the former case, the devicecan allow the corresponding event to be available in applications inwhich a callback routine is registered through a general method. In thelatter case, the event may include URLs of updated signaling files andthe device can acquire the updated signaling files through the URLs.

If the audio WM disappears, the device can stop presentation ofsupplementary and secondary content because the device may interpretdisappearance of the audio WM as channel change to a new service whichdoes not provide supplementary content.

When the server field of the audio WM is changed or interval fields arenot consecutive, the device can stop presentation of supplementary andsecondary content. Further, the device may request a new recovery fileand newly start a recovery process.

According to an embodiment, in the case of the ROUT/DASH service, thepresentation time of the recovery file may be relative to the mediapresentation timeline of the current MPD of the service. In the case ofthe MMT service, the presentation time of the recovery file may be anNPT (Non-Productive Time). The presentation time may be represented asTAI irrespective of the transport protocol according to an embodiment.

A method of acquiring signaling information using a video WM will bedescribed below.

Here, although description is based on the assumption that media timeinformation is delivered in a channel ID WM message, the media timeinformation may be delivered in a content ID WM message or theaforementioned presentation_time_message( ) according to an embodiment.The media time information may need to appear through a WM payload atleast once per 1.5 seconds.

A device can acquire a channel ID message and a WM message having mediatime information through video WMs. In addition, the device can acquiresignaling URIs through the uri_message( ) of a video WM.

The device can acquire signaling files using the obtained signalingURLs. The signaling files may have a multi-part MIME message form. Thismethod may be the same as the method of using a signaling URL of arecovery file acquired using an audio WM.

The device can access supplementary content using the signaling filesand present the same. In the process, the time information of therecovery file can be used for synchronization between the supplementarycontent and audio/video received from a cable.

Each signaling file may have a “valid from” attribute, a “valid until”attribute and/or a “next URL” attribute related thereto. The “validfrom” attribute and/or the “valid until” attribute define validintervals of the corresponding signaling file. The “next URL” attributeindicates the URL of the corresponding signaling file of the schedulednext version. The device can acquire scheduled update of thecorresponding signaling file as necessary through the attributes.

Dynamic events may appear in a dynamic event WM message of a video WM.Such events may be intended for applications being executed in arun-time environment. Furthermore, the events may signal availability ofunscheduled update for a signaling file. In the former case, the devicecan allow the corresponding event to be available in applications inwhich a callback routine is registered through a general method. In thelatter case, the event may include URLs of updated signaling files andthe device can acquire the updated signaling files through the URLs. Thesignaling files may have the same format as files acquired usingsignaling URLs. That is, they can have the “valid from” attribute,“valid until” attribute and/or “next URL” attribute.

Whenever the video WM disappears for an interval of 1.5 seconds, thedevice can stop presentation of supplementary content because the devicemay interpret disappearance of the video WM as channel change to a newservice which does not provide supplementary content.

Whenever a WM payload having a channel ID WM message disappears for aninterval of 1.5 seconds or when the WM payload appears with a newBSID-channel number combination or a media time has discontinuity, thedevice can stop presentation of supplementary content. In addition, thedevice can newly start a signaling acquisition process.

According to an embodiment, in the case of the ROUTE/DASH service, apresentation time provided through the video WM may be relative to themedia presentation timeline of the current MPD of the service. In thecase of the MMT service, the presentation time may be an NPT(Non-Productive Time). The presentation time may be represented as TAIirrespective of the transport protocol according to an embodiment.

A recovery file format according to the illustrated embodiment will bedescribed. A recovery file may be supplementary content which isdelivered from a server upon request using the aforementioned WMinformation. The recovery file may be called recovery data.

The recovery file may have a JSON format or an XML format according toan embodiment. There may be a unique recovery file for each vpl WMmessage (vpl payload) according to an embodiment.

The recovery file according to the illustrated embodiment may include athisComponent element, a querySpread element, an otherComponent element,a contentID element, a sourceID element and/or a service element.

The thisComponent element may include information about a mediacomponent in which a vpl payload carrying a serverCode and anintervalCode is detected. This element may include a serverCode field,an intervalCode field and/or a componentDescription field.

The serverCode field can indicate a server code value used for query forthe corresponding recovery file. The intervalCode field can indicate aninterval code value used for query for the corresponding recovery file.The componentDescription field can include a component description of aservice component indicated by the thisComponent element.

The querySpread element can indicate a time for which a device can querythe dynamic event server. This information may be intended fordistributing queries of devices.

The otherComponent element can include information about componentscarrying a matching vpl payload which are audio/video components otherthan the current component indicated by the thisComponent element. Forexample, when the current component (component represented by thethisComponent element) is an audio component and has a specific vplpayload, the otherComponent element can describe video componentsincluding linked video WMs having the same vpl payload.

The contentID element can indicate a content ID. For example, thiselement can refer to an ID of content in which the corresponding WM isdetected. This element may include a type field, a cid field, avalidFrom field and/or a validUntil field.

The type field can indicate whether the corresponding content ID is anEIDR type or an Ad-ID type. The cid field can include an EIDR typecontent ID or an Ad-ID type content ID according to indicated type. Thatis, this field can include a content ID. The validFrom field canindicate a start point from which a content ID is valid. The validUntilfield can indicate a last point at which the content ID is valid.

The sourceID element can indicate an ID of a service/content source.This element may include a country field, a bsid field, a majorChannelNofield and/or a minorChannelNo field.

The country field can indicate a country code related to anadministrative entity assigned a corresponding BSID. The bsid field canindicate the ID of the entire broadcast stream. The majorChannelNo fieldand the minorChannelNo field can indicate major/minor channel numbers ofthe corresponding service.

The service element can indicate information about the correspondingservice. For example, this element can include information about aservice in which the corresponding WM is detected. This element mayinclude a serviceId field, an sltSveSeqNum field, an slsProtocol field,an slsMajorProtocolVersion field, an slsMinorProtocolVersion fieldand/or an svcInetUrl field.

The serviceId field can indicate the ID of the corresponding service.The ID may be unique in a BSID scope. The sltSveSeqNum field canindicate the version of service information in the corresponding table.The value of this field can increase for each new version of serviceinformation in the recovery file. The slsProtocol field can indicate atransport protocol used to deliver service layer signaling (SLS) of thecorresponding service. The slsMajorProtocolVersion field and theslsMinorProtocolVersion field can indicate major/minor version numbersof a transport protocol used to deliver SLS of the correspondingservice. The svcInetUrl field can include URL related information foracquiring a signaling file of the corresponding service. This elementmay include a urlType field and/or a urlValue field. The urlType fieldcan indicate the type of an included URL. Types of URLs which can beincluded may include a signaling server for SLS delivery, an EGS serverfor ESG delivery, a usage reporting server, and a dynamic event HTTPserver/WebSocket server for dynamic event delivery. The urlValue fieldcan include URL information for accessing the signaling file of thecorresponding service.

FIG. 50 is a diagram illustrating a recovery file format according toanother embodiment of the present invention.

A recovery file of the illustrated embodiment may include an@intervalCode attribute, an @anchorCode attribute, a Choice element, an@querySpread attribute, an @bsid attribute and/or a Service element.

The @intervalCode attribute is interval code information in a vplpayload (audio WM and video WM) and may be the same as theaforementioned intervalCode field.

The @anchorCode attribute can indicate an interval code used as ananchor for a presentation time. This attribute may be intended formapping between the first vpl payload of WM segments and a presentationtime thereof.

The Choice element can provide presentation time related information.This element may include an @mpdPresentationTime attribute and/or an@mmtPresentationTime attribute.

The @mpdPresentationTime attribute can indicate an MPD presentation timecorresponding to the @anchorCode. This may be intended to indicate thepresentation time of an MPD which is a basis for presentation time. Thisattribute may further include an @mpdId attribute. This attribute may bean ID for identifying an MPD which is a basis of timeline.

The mmtPresentationTime attribute can indicate an MMT presentation timecorresponding to the @anchorCode. This can be represented as a 64-bitNTP. The @querySpread attribute may be the same as the aforementionedquerySpread field. The @bsid attribute may be the same as theaforementioned bsid field.

The Service element indicates service related information and may be thesame as the aforementioned Service element. This element may include an@serviceId attribute, an @serviceInfoVersion attribute, an @protectedattribute, an @majorChanNum attribute, an @minorChanNum attribute, an@serviceCategory attribute, an @serviceName attribute, an@slsProtocolType attribute, an @slsProtocolVersion attribute, an@serviceLanguage attribute, an @serviceCapabilities attribute, an@signalingUri attribute, a choice element and/or an @esgUri attribute.

The @serviceId attribute is a service ID and may be the same as theaforementioned serviceId field. The @serviceInfoVersion attribute is aversion of service information and may be the same as the aforementionedsltSvcSeqNum field. The @protected attribute indicates that access of atleast one essential component of a service is controlled when set to“true” and is set to “false” in other cases. The @majorChanNum attributeand the @minorChanNum attribute may be the same as the aforementionedmajorChannelNo field and the minorChannelNo field.

The @serviceCategory attribute can indicate a service category of thecorresponding service. This attribute can indicate a linear A/V servicewhen set to 0x01, a linear audio service (audio only) when set to 0x02,and an app-based service when set to 0x03. Other values may be reservedfor future use. The @serviceName attribute can indicate the service nameof the corresponding service. The @slsProtocolType attribute may be thesame as the aforementioned slsProtocol field. The @slsProtocolVersionattribute is the sum of the aforementioned slsMajorProtocolVersion fieldand slsMinorProtocolVersion field and can indicate a version number of atransport protocol used to deliver SLS of the corresponding service. The@serviceLanguage attribute can indicate a predominate language in thecorresponding service.

The @serviceCapabilities attribute can indicate capabilities necessaryfor significant rendering of the corresponding service. This can berepresented by a capability code according to an embodiment. The@signalingUri attribute can indicate the URL of a signaling server foracquiring signaling files.

The choice element may include an @eventSocketUri attribute and/or an@eventRequestUri attribute. The former can include URI information forweb socket connection for dynamic events and the latter can include aURI for a request of a dynamic event. The @esgUri attribute can includethe URL of an ESG server for ESG delivery.

A device knows the present wall clock time at the beginning of an audioWM and a wall clock interval between interval codes (1.5 seconds).Accordingly, the @MPDpresentationTime field can provide mapping betweenan MPD presentation timeline and a local time (wall clock in the device)to the device in a service based on ROUTE/DASH. The @MMTpresentationTimefield can provide an offset between an NPT presentation timeline ofcontent and a local time (wall clock in the device) to the device in aservice based on MMT.

Accordingly, supplementary content can be synchronized with audio/videofrom an MVPD in the aforementioned cases. Such synchronization can beperformed by switching the presentation time of supplementary contentderived from an MPD or MMT signaling to a local wall clock time.

The aforementioned slsProtocol field and @slsProtocolType attribute canindicate that the ROUTE protocol is used when set to 0x01 and indicatethat the MMT protocol is used when set to 0x02. Other values may bereserved for future use.

The 4 MSBs (Most Significant Bits) of the @slsProtocolVersion attributecan indicate a major protocol version of the ROUTE protocol indicated bythe @slsProtocolType attribute. The 4 LSBs (Least Significant Bits) ofthe @slsProtocolVersion attribute can indicate a minor protocol versionof the ROUTE protocol indicated by the @slsProtocolType attribute. Here,the major version number may be 0x01 and the minor version number may be0x0 according to an embodiment.

When the @signalingUri attribute is used to request combined sets ofsignaling files, the signaling files can be delivered in the form of amulti-part MIME message. The MIME message can include a metadataenvelope which references all files of combined sets. The metadataenvelope may be followed by signaling files in the order of referencingby the metadata envelope. That is, a file itself may not beencapsulated.

When an item element of the metadata envelope does not include avalidUntil attribute, this represents that the corresponding file doesnot have a scheduled expiration date/time. When the item element of themetadata envelope includes the validUntil attribute, this attribute caninclude a nextUrl sub-element. The nextUri sub-element can provide a URLfor the next version of the corresponding file. That is, a URL for afile of a version valid after scheduled expiration of the current filecan be provided. The validUntil attribute can indicate a value relativeto the current MPD and NPT timeline according to an embodiment.

When the @eventRequestUri attribute is used for query of a dynamicevent, the query can include a media presentation time as a query term.The media presentation time may correspond to a local time indicated bythe query flag as a time when the dynamic event is available. In thecase of a ROUTE/DASH based service, the @eventRequestUri attribute mayfurther include MPD ID information. The MPD ID can form a presentationtimeline reference for a presentation time.

When the @eventSocketUri attribute is used to set up a connection for adynamic event, a query can also include a media presentation time as aquery term. The presentation time may correspond to a local time atwhich the query is performed.

The event acquired through the query may have the form of theaforementioned event WM message according to an embodiment.

When the @esgUri is used for query of ESG data, a query language can bedefined in a form conforming to OMA BCAST service guide standards.

FIG. 51 is a diagram illustrating a broadcast content processing methodaccording to an embodiment of the present invention.

The broadcast content processing method according to an embodiment ofthe present invention may include a step of receiving broadcast contentfrom an external input source, a step of extracting audio or videowatermarks from the broadcast content, a step of requestingsupplementary content of the broadcast content with a URL generated fromwatermarks and/or a step of acquiring the supplementary content from aserver.

A reception unit of a receiver can receive broadcast content from anexternal input source. Here, the external input source may refer to anMVPD such as an STB, a cable or a satellite. The broadcast content isderived from broadcast streams and may be uncompressed AV content. Thebroadcast content may include audio and/or video components, and audioWMs and video WMs may be embedded in the components. Broadcast streamsmay be received by the external input source according to the ROUTEprotocol or the MMT protocol. Details have been described above.

An extractor of the receiver can extract at least one of the audio WMsand the video WMs from the broadcast content. A network interface of thereceiver can request supplementary content from a specific server with aURL generated from the audio or video WMs. The supplementary content isrelated to the broadcast content and has been described in detail above.The supplementary content may correspond to the aforementioned recoveryfile. The URL may be derived from a server code or a server URLinformation of a WM payload, or a uri_message( ) of the video WM. Thenetwork interface can acquire the supplementary content from the server.The audio/video WMs can be used for delivery of auxiliary data relatedto the broadcast content.

In the present embodiment, one of the video watermarks can include apayload having a fragment of a first message. Here, the first messagemay correspond to the aforementioned vpl WM message. Here, the fragmentof the first message is a fragment of the aforementioned vpl WM messageor vpl payload and may correspond to a vpl WM message block. The firstmessage can include domain type information, server information and/orinterval information. This information can correspond to theaforementioned domain type information, server code and interval code.

In a broadcast content processing method according to another embodimentof the present invention, the domain type information can specify aninformation type of the first message, the server information canidentify a server for supplementary content acquisition, and theinterval information can identify an interval of a video component inwhich video WMs are embedded. Such information has been described indetail above.

In a broadcast content processing method according to another embodimentof the present invention, one of the audio watermarks can include apayload having the same information as the first message. That is, anaudio WM and a video WM can be linked to include a vpl payload havingthe same information according to an embodiment. The first message, thatis, one fragment of the vpl payload can be delivered by theaforementioned video WM. The audio watermark can be time-aligned withthe video watermark having the fragment of the first message. Linkagebetween audio/video WMs has been described in detail above.

In a broadcast content processing method according to another embodimentof the present invention, the first message may further include queryinformation which signals when event signaling is available from aserver. The query information can signal whether event signaling isavailable from a server. The query information may correspond to theaforementioned query_flag. Event signaling can correspond to theaforementioned event information which initiates operation of anapplication.

A broadcast content processing method according to another embodiment ofthe present invention may further include a step of requesting a firstevent message for event signaling of a first event from a server usingthe query information, and a step of receiving the first event messagefrom the server. Here, the first event may correspond to theaforementioned dynamic event. The first event message may refer to theaforementioned event WM message. The present embodiment may correspondto an embodiment of acquiring event signaling information through arequest for the server using the query flag of the audio WM.

In a broadcast content processing method according to another embodimentof the present invention, the first event message may include a protocoltype field, an event ID field, a time field or a data field. Thesefields may respectively correspond to the delivery_protocol_type field,presentation time field/presentation_time_ms field, id field and datafield in the aforementioned dynamic₁₃ event_message( ). The protocoltype field can indicate a transport protocol of a broadcast service towhich the first event is applied. The event ID field can identify thefirst event. The time field can indicate a presentation time of thefirst event. Here, the presentation_time field and/or thepresentation_time_ms field can be used. The data field can include datanecessary for the first event.

In a broadcast content processing method according to another embodimentof the present invention, a video watermark payload may further includea fragment of the second event message for event signaling of a secondevent. Here, the second event may correspond to the aforementioneddynamic event. The second event message may refer to the aforementionedevent WM message. Here, the video WM payload is a payload indulging amessage block of the aforementioned vpl message and may further includean event WM message block according to an embodiment. The second eventmessage may include a protocol type field, an event ID field, a timefield or a data field. These fields may respectively correspond to thedelivery_protocol_type field, presentation timefield/presentation_time_ms field, id field and data field in theaforementioned dynamic_event_message( ). These fields can providesignaling information about the second event. The present embodiment maycorrespond to an embodiment of acquiring event signaling informationthrough an event WM message of a video WM.

In a broadcast content processing method according to another embodimentof the present invention, the supplementary content may include aninterval code used to request service information and/or supplementarycontent. The supplementary content may correspond to the aforementionedrecovery file. The service information and the interval code maycorrespond to the aforementioned Service element and the intervalCodefield.

The service information may include an SLS (Service Layer Signaling)protocol field, an SLS protocol version field, an SLS URL field or anESG (Electronic Service Guide) URL field. These fields may correspond tothe slsProtocol field, the slsMajorProtocolVersion field, theslsMinorProtocolVersion field and the svcInetUrl field of the Serviceelement. The SLS protocol field can indicate a transport protocol usedto deliver SLS, the SLS provides service level signaling information ofa broadcast service, and the SLS protocol version field can indicate theversion of the transport protocol used to deliver the SLS. The SLSprotocol version field may refer to the slsMajorProtocolVersion field,the slsMinorProtocolVersion field or both thereof. The SLS URL field canprovide a URL for accessing the SLS of the broadcast service, and theESG URL field can provide a URL for accessing an ESG of the broadcastservice. These fields may correspond to the svcInetUrl field.

In a broadcast content processing method according to another embodimentof the present invention, at least one of video WMs may include a videoWM payload, and the video WM payload may include at least one messageblock having the same information as an audio WM payload. As describedabove, the video WM payload can allocate and include a space for theaudio WM payload.

In a broadcast content processing method according to another embodimentof the present invention, supplementary content may includethisComponent information and/or otherComponent information. ThethisComponent information can describe information about a servicecomponent in which a WM payload used to request the supplementarycontent is detected. The supplementary content may be requested usingserver information/interval information of the WM payload. TheotherComponent information can describe a component having a WM payloadcorresponding to the WM payload among components other than thecomponent described by the thisComponent information. The WM payload maybe a video or audio WM.

A broadcast content processing method according to another embodiment ofthe present invention may further include a step of recognizing that, bythe extractor of the receiver, one of audio or video WMs hasdisappeared, and a step of extracting other WMs which have notdisappeared from the corresponding component. This may be intended torecognize that a video or audio WM has disappeared and access a WMhaving the same payload to perform seamless normal operation when an ESGis displayed on a screen or muting is performed.

A broadcast content processing method (at a transmitting side) accordingto an embodiment of the present invention will be described. This methodis not shown.

The broadcast content processing method (at a transmitting side)according to an embodiment of the present invention may include a stepin which a service data generation module generates a broadcast servicehaving video/audio components, a step in which a WM embedding moduleembeds video/audio WMs in the video/audio components, a step in whichthe service data generation module generates signaling informationrelated to the broadcast service and/or a step in which a transmissionunit transmits broadcast service data and signaling information to anMVPD. The broadcast content processing method may further include a stepin which the service data generation module generates supplementarycontent with respect to the broadcast service and delivers thesupplementary content to a supplementary content server and the WMembedding module embeds information related to a server and thesupplementary content in a WM according to an embodiment. Transmissionof the service data may be performed according to ROUTE or MMT protocol.

The broadcast content processing methods (at a transmitting side)according to embodiments of the present invention may correspond to theabove-described broadcast content processing methods according toembodiments of the present invention. The broadcast content processingmethods may have embodiments corresponding to the above-describedembodiments of the broadcast content processing methods.

The above-described steps may be omitted or replaced by other stepsperforming similar/identical operations according to embodiments.

FIG. 52 is a diagram illustrating a broadcast content processing deviceaccording to an embodiment of the present invention.

The broadcast content processing device according to an embodiment ofthe present invention may include the aforementioned reception unit,extractor and/or network interface. Blocks and modules thereof have beendescribed above. The broadcast content processing device according to anembodiment of the present invention and internal modules/blocks thereofcan perform the above-described embodiments of the broadcast contentprocessing methods of the present invention.

A broadcast content processing device (at a transmitting side) accordingto an embodiment of the present invention will be described. The deviceis not shown. The broadcast content processing device (at a transmittingside) according to an embodiment of the present invention may includethe aforementioned service data generation module, WM embedding moduleand/or transmission unit. Blocks and modules thereof have been describedabove. The broadcast content processing device (at a transmitting side)according to an embodiment of the present invention and internalmodules/blocks thereof can perform the above-described embodiments ofthe broadcast content processing methods (at a transmitting side) of thepresent invention.

The internal blocks/modules of the above described devices may behardware elements provided inside/outside of the devices which performcontinuous processes stored in a memory. The above-described modules maybe omitted or replaced by other modules performing similar/identicaloperations according to embodiments.

Modules or units may be processors executing consecutive processesstored in a memory (or a storage unit). The steps described in theaforementioned embodiments can be performed by hardware/processors.Modules/blocks/units described in the above embodiments can operate ashardware/processors. The methods proposed by the present invention canbe executed as code. Such code can be written on a processor-readablestorage medium and thus can be read by a processor provided by anapparatus.

While the embodiments have been described with reference to respectivedrawings for convenience, embodiments may be combined to implement a newembodiment. In addition, designing computer-readable recording mediastoring programs for implementing the aforementioned embodiments iswithin the scope of the present invention.

The apparatus and method according to the present invention are notlimited to the configurations and methods of the above-describedembodiments and all or some of the embodiments may be selectivelycombined to obtain various modifications.

The methods proposed by the present invention may be implemented asprocessor-readable code stored in a processor-readable recording mediumincluded in a network device. The processor-readable recording mediumincludes all kinds of recording media storing data readable by aprocessor. Examples of the processor-readable recording medium include aROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical datastorage device and the like, and implementation as carrier waves such astransmission over the Internet. In addition, the processor-readablerecording medium may be distributed to computer systems connectedthrough a network, stored and executed as code readable in a distributedmanner.

Although the preferred embodiments of the present invention have beendisclosed for illustrative purposes, those skilled in the art willappreciate that various modifications, additions and substitutions arepossible, without departing from the scope and spirit of the inventionas disclosed in the accompanying claims. Such modifications should notbe individually understood from the technical spirit or prospect of thepresent invention.

Both apparatus and method inventions are mentioned in this specificationand descriptions of both the apparatus and method inventions may becomplementarily applied to each other.

Those skilled in the art will appreciate that the present invention maybe carried out in other specific ways than those set forth hereinwithout departing from the spirit and essential characteristics of thepresent invention. Therefore, the scope of the invention should bedetermined by the appended claims and their legal equivalents, not bythe above description, and all changes coming within the meaning andequivalency range of the appended claims are intended to be embracedtherein.

In the specification, both the apparatus invention and the methodinvention are mentioned and description of both the apparatus inventionand the method invention can be applied complementarily.

Various embodiments have been described in the best mode for carryingout the invention.

The present invention is applied to broadcast signal providing fields.

Various equivalent modifications are possible within the spirit andscope of the present invention, as those skilled in the relevant artwill recognize and appreciate. Accordingly, it is intended that thepresent invention cover the modifications and variations of thisinvention provided they come within the scope of the appended claims andtheir equivalents.

What is claimed is:
 1. A method processing a broadcast content, themethod comprising: receiving a broadcast signal; demodulating thebroadcast signal including time interleaved data, wherein the timeinterleaved data is interleaved by a Time interleaved (TI) blockincluding one or more Forward Error Correction (FEC) blocks and one ormore virtual FEC blocks; displaying a broadcast content of a broadcastservice in the demodulated broadcast signal on a display screen; anddetecting a first user input, in response to the first user input thatis to display an Electronic Service Guide (ESG): displaying the ESGoverlaid on the broadcast content being displayed on the display,screen; determining whether information for controlling execution of anapplication, that is embedded in an audio component of the broadcastcontent, is received; and controlling the execution of the applicationbased on the determination; or in response to the first user input thatindicates a mute mode to mute sound of the broadcast content:determining whether information for controlling execution of theapplication, that is embedded in a video component of the broadcastcontent, is received; and controlling the execution of the applicationbased on the determination.
 2. The method of claim 1, furthercomprising: executing the application while displaying the broadcastcontent.
 3. The method of claim 2, wherein controlling the execution ofthe application based on the determination further includes: in responseto determining that the information for controlling execution of theapplication is not received, immediately stopping the execution of theapplication so that the application is started over after the ESGdisappears from the display screen; and in response to determining thatthe information for controlling execution of the application isreceived, temporarily suspending the execution of the application sothat the application is resumed after the ESG disappears from thedisplay screen.
 4. The method of claim 2, wherein controlling theexecution of the application based on the determination furtherincludes: in response to determining that the information forcontrolling execution of an application is not received, immediatelystopping the execution of the application so that the application isstared over after receiving a second user input to turn off the mutemode; and in response to determining that the information forcontrolling execution of the application is received, temporarilysuspending the execution of the application so that the application isresumed after receiving a second user input to turn off the mute mode.5. The method of claim 1, wherein controlling the execution of theapplication based on the determination further includes: in response todetermining that the information for controlling execution of theapplication requests to execute the application which is not executed:determining whether second information for controlling execution of theapplication embedded in an audio component subsequent to the audiocomponent is received, in response to determining that the secondinformation for controlling execution of the application is notreceived, immediately executing the application with the ESG; and inresponse to determining that the second information for controllingexecution of the application is received, executing the applicationafter the ESG disappears from the display screen.
 6. The method of claim1, wherein controlling the execution of the application based on thedetermination further includes: in response to determining that theinformation for controlling execution of the application requests toexecute the application which is not executed: determining whethersecond information for controlling execution of the application embeddedin a video component subsequent to the video component, is received, inresponse to determining that the second information for controllingexecution of the application is not received, immediately executing theapplication regardless of the mute mode; and in response to determiningthat the second information for controlling execution of the applicationis received, executing the application after receiving the second userinput to turn off the mute mode.
 7. The method of claim 1, wherein thebroadcast service is delivered to an external input source according toeither a Real-Time Object Delivery over Unidirectional Transport (ROUTE)protocol or an MPEG Media Transport (MMT) protocol.
 8. The method ofclaim 1, wherein the information for controlling execution of theapplication embedded in the audio component is same as the informationfor controlling execution of the application embedded in the videocomponent.
 9. The method of claim 1, wherein the application is eitheran audible application or a visible application.
 10. A device processinga broadcast content, the device comprising: a receiver configured toreceive a broadcast signal; a demodulator configured to demodulate thebroadcast signal including time interleaved data, wherein the timeinterleaved data is interleaved by a Time Interleaved (T1) blockincluding one or more Forward Error Correction (FEC) blocks and one ormore virtual FEC blocks; a display screen configured to display abroadcast content of a broadcast service in the demodulated broadcastsignal; a user interface managing processor configured to detect a firstuser input; an extractor configured to extract information forcontrolling execution of an application embedded in an audio componentof the broadcast content, and information for controlling execution ofthe application embedded in a video component of the broadcast content;and a controller configured to control execution of the application,wherein: in response to the first user input that is to display anElectronic Service Guide (ESG), the display screen is further configuredto display the ESG overlaid on the broadcast content being displayed onthe display screen, the extractor is configured to determine whether theinformation for controlling execution of the application embedded in theaudio component is received, and the controller is configured to controlthe execution of the application based on the determination; or inresponse to the first user input that indicates a mute mode to mutesound of the broadcast content, the extractor is further configured todetermine whether the information for controlling execution of theapplication embedded in the video component is received, and thecontroller is configured to control the execution of the applicationbased on the determination.
 11. The device of claim 10, wherein theapplication is executed while displaying the broadcast content.
 12. Thedevice of claim 11, wherein: in response to determining that theinformation for controlling execution of the application is notreceived, the controller is further configured to immediately stop theexecution of the application so that the application is started overafter the ESG disappears from the display screen, and in response todetermining that the information for controlling execution of theapplication is received, the controller is further configured totemporarily suspend the execution of the application so that theapplication is resumed after the ESO disappears from the display screen.13. The device of claim 11, wherein: in response to determining that theinformation for controlling execution of the application is notreceived, the controller is further configured to immediately stop theexecution of the application so that the application is stared overafter receiving a second user input to turn off the mute mode, and inresponse to determining that the information for controlling executionof the application is received, the controller is further configured totemporarily suspend the execution of the application so that theapplication is resumed after receiving a second user input to turn offthe mute mode.
 14. The device of claim 10, wherein: in response todetermining that the information for controlling execution of theapplication requests to execute the application which is not executed,the extractor is further configured to determine whether secondinformation for controlling execution of the application embedded in anaudio component subsequent to the audio component, is received, inresponse to determining that the second information for controllingexecution of the application is not received, the controller is furtherconfigured to immediately execute the application with the ESG, and inresponse to determining that the second information for controllingexecution of the application is received, the controller is furtherconfigured to execute the application after the ESG disappears from thedisplay screen.
 15. The device of claim 10, wherein: in response todetermining that the information for controlling execution of theapplication requests to execute the application which is not executed,the extractor is further configured to determine second information forcontrolling execution of the application embedded in a video componentsubsequent to the video component, is received, in response todetermining that the second information for controlling execution of theapplication is not received, the controller is further configured toimmediately execute the application regardless of the mute mode, and inresponse to determining that the second information for controllingexecution of the application is received, the controller is furtherconfigured to execute the application after receiving the second userinput to turn off the mute mode.
 16. The device of claim 10, wherein thebroadcast service is delivered to an external input source according toeither a Real-Time Object Delivery over Unidirectional Transport (ROUTE)protocol or an MPEG Media Transport (MMT) protocol.
 17. The device ofclaim 10, wherein the information for controlling execution of theapplication embedded in the audio component is same as the informationfor controlling execution of the application embedded in the videocomponent.
 18. The device of claim 10, wherein the application is eitheran audible application or a visible application.